• Technology Consultant.
  • Software Developer.
  • Musician.
  • Burner.
  • Game Master.
  • Non-theistic Pagan.
  • Cishet White Male Feminist.
  • Father.
  • Fountain Maker.
  • Aquarium Builder.
  • Hamster Daddy.
  • Resident of Colorado.
  • Anti-Capitalist.
  • Hackerspace Regular.
  • Traveler of the American West.
  • 6 Posts
  • 296 Comments
Joined 2 years ago
cake
Cake day: June 7th, 2023

help-circle


  • Sure, you can. There are even some very legitimate use cases.

    But for your goals as you’ve defined them, you’re kind of just creating extra layers of complication for yourself. Sure, you could store all your data in MySQL AND Mongo and write to both with every write operation. If you do this, only read from ONE. You could write methods to toggle back and forth for which one you’re reading from but WHY?

    A common usecase for this kind of thing is using Mongo as an object cache. You pull some common class of thing out of SQL that involves like

    SELECT

    g.group_id,

    g.name,

    g.blahblahblah,

    COALESCE(json_agg(u.*) FILTER (WHERE u.user_id IS NOT NULL), '[]') AS moderators

    FROM groups g

    LEFT JOIN group_moderators gm ON g.group_id = gm.group_id

    LEFT JOIN users u ON gm.user_id = u.user_id

    GROUP BY g.group_id, g.name, g.blahblahblah;

    Now you have this complex query returning your group object, with this JSON blob in it that stores all the moderator IDs. Now… what you can do is cache that in Mongo. When the API asked for a group by group_id, it can check to see if it exists in Mongo and if it does it can just grab that and return it to the user. If it doesn’t, it can ask SQL for it, then cache that in Mongo. If a new moderator gets added, that operation can refresh the Mongo cache.

    Now WHY would you do this?

    • Maybe Mongo has better performance.
    • Maybe you’re trying to make less requests to your SQL server.
    • Maybe Mongo runs locally on the same host as your endpoint, but the SQL server is remote and you want to use less bandwidth (this can cause some silly problems when you have multiple end points just sayin - but those problems have known simple solutions).
    • You think its cool and you’re down to do the extra work.
    • Any combination of these.

    Now, IMO Redis is a superior tool for this, but I HAVE heard of people using Mongo.

    But this isn’t really your usecase is it?

    It could be a good learning exercise to set up your database reads and writes to use both MySQL and MongoDB. I’d actually recommend using either PostGres or MariaDB over MySQL… You’re going to encounter PostGres a LOT more often in the wild, so getting comfortable with it is a good use of your time. If you REALLY want MySQL without the taint of Oracle, Maria has got you covered.

    Either way writing methods to read / write from both databases will certainly help you learn about both kinds of databases. Just make sure that when your site is actually live, you’re treating one database as the source of truth and the other as a mirror.

    Given the scope of your project, I bet you’ll find both databases are perfectly adequate to meet your needs. But there are reasons many people have strong preferences for truely relational databases and it sounds like you’re kind of discovering that on your own.

    But, you should probably just not complicate things for yourself at this point

    You’re at a point where you can say

    “I’m gonna just use Mongo, but I want to build things in such a way that it’s possible to change my mind later” just write your data handler class in such a way that you can save yourself pain down the road.

    For example, you might have a data::getObject method() or a data::createUser() method. If you write this intelligently now, you can even preemptively add something like

    if(database==="mongo"){ <-do stuff to make a new user in mongo-> }else{ <-Throw an unknown database exception-> }

    Now, when you come back to switch databases in the future, all you have to do is

    if(database==="mongo"){ <-do stuff to make a new user in mongo-> elseif(database==="postgres"){ <-do stuff to make a new user in postgres-> else{ <-Throw an unknown database exception-> }

    But an even better and more secure way to do this is to make your data class ignorant of what kind of database you’re using (at the method level) and just define your schema. If you’re using FastAPI (I personally loath working in Python, eff FastAPI, I roll my own APIs in Rust, Lua and PHP) it’s kind of set up to get you started on this quickly. You probably should use it, given where you’re at. There’s reasons it’s the most popular tool out there for doing this kind of thing and will teach you a lot about how to structure an API.

    https://www.fastapitutorial.com/blog/pydantic-schemas-in-fastapi/

    https://hevodata.com/learn/fastapi-mongodb/



  • My experience working with a vibe coder hired by one of our clients is actually that’s it great for MAXIMUM VP, as in a viable product made up of just under 40k lines of typescript, plus 90+ Node libraries for an app that amounts to a login page, a Square payment gateway and a user settings page (it’s seriously just a signup page for a coastguard and weather alerts service that the rest of our team built in Python and Rust). It crashes if it can’t talk to a database server that hosts no actual databases. It crashes if it doesn’t have the Square API secrets as envars, but the LLM also hard coded them into the API calls. It actually crashes if you try to run it any way other than “npm run dev” (so I srsly set up a service that runs it as npm run dev, as the ubuntu user).








  • This is extremely possible and I have done a lot of stuff like it (I set up my first home built Linux firewall over 20 years ago). You do want to get some kind of multiport network card (or multiple network cards… usb -> ethernet adapters can do OK filling in in a pinch). It also gives you a lot of power if you want to do specific stuff with specific connections (sub netting, isolation of specific hosts, etc).

    There’s a lot of ways to do it, but the one I’m most familiar with is just to use IP tables.

    The very first thing you want to do is open up /proc/sys/net/ipv4/ip_forward and change the 0 to a 1 to turn on network forwarding.

    You want to install bridge-utils and isc-dhcp-server (or some other DHCP server). Google or get help from an LLM to configure them, because they’re powerful and there’s a lot of configs. Ditto if you want it to handle DNS. But basically what you’re going to do (why you need bridge-utils) is you’re going to set up a virtual bridge interface and then add all the various NICs you want on your LAN side into it (or you can make multiple bridges or whatever… lots of possibilities).

    Your basic iptables rule is going to be something like

    iptables -t nat -A POSTROUTING -o enp1s0 -j MASQUERADE, but again there’s lots of possible IP tables rules so read up on those.





  • I’m working with a team where my business partner and I are external consultants, but they also have internal developers (who are mostly very junior and need hand holding with things like using git).

    Anyway, the CEO (without talking to us first) hired a pure vibe coder with no software engineering experience to build the user interface. Super nice guy, super easy to work worth, super eager to learn but OH MY GOD THIS CODE.

    A lot of my work is / has been in cybersecurity (mostly for the space industry / NASA adjacent projects, but also less recently for start ups and fortune 500 companies). This app is the worst I’ve ever seen. The AI writes things SO weirdly. 30k lines of typescript to do something we could have done in 6k. Reams of dead code. Procedural code to do repeatable tasks instead of functions / classes (10 different ways of doing the same thing). API keys / data base credentials committed to git. API Keys stored in .env but then ALSO just hardcoded into the actual API calls.

    AND no. At the end of the day, it wasn’t cheaper or faster than it would have been to hire us to do it right. And the tech debt now accumulated to secure / maintain this thing? Security is a long term requirement, we’re bringing a buddy of mine in to pentest this thing next week, I expect him to find like 10-12 critical vulns. Wow.

    tl;dr: If a project requires security, stability, auditability, or the need to quickly understand how something works / why something happens, DON’T vibe code it. You won’t save money OR time in the long run. If you’re project DOESN’T need any of those things (and never will), then by all means I guess, knock yourself out.




  • We tried to build systems that perform a kind of basic, rudimentary, extremely power intensive and inefficient mimicry of how (we think maybe) brain cells work.

    Then that system lies to us, makes epic bumbling mistakes, expresses itself with extreme, overconfidence, and constantly creatively misinterprets simple instructions. It recognizes patterns that aren’t there, and regurgitates garbage information that it picks up on the internet.

    Hmmm… Actually, maybe we’re doing a pretty good job of making systems that work similarly to the way brain cells work…