• 1 Post
  • 27 Comments
Joined 1 year ago
cake
Cake day: June 19th, 2023

help-circle


  • It cannot “analyze” it. It’s fundamentally not how LLM’s work. The LLM has a finite set of “tokens”: words and word-pieces like “dog”, “house”, but also like “berry” and “straw” or “rasp”. When it reads the input it splits the words into the recognized tokens. It’s like a lookup table. The input becomes “token15, token20043, token1923, token984, token1234, …” and so on. The LLM “thinks” of these tokens as coordinates in a very high dimensional space. But it cannot go back and examine the actual contents (letters) in each token. It has to get the information about the number or “r” from somewhere else. So it has likely ingested some texts where the number of "r"s in strawberry is discussed. But it can never actually “test” it.

    A completely new architecture or paradigm is needed to make these LLM’s capable of reading letter by letter and keep some kind of count-memory.


  • We long left the era where we “own” things that we buy. As everything is a computer now it has become very simple to control stuff that remotely that was working on its own before.

    So the answer to “why would <CORPORATION> do this” is simply: “Because they can”.

    Every tiny decision is guided by increasing profit. No matter the side effects (short or long term ). Because with many shareholders administering pressure to maximize profits there’s only one way to go (even if it’s a dumb and shortsighted decision) maximizing profits NOW. If you are not doing that because you can see that increasing profits now will hurt profits in the future then you are hindering the project. You have to increase profits now, because if you are not then your competitor is doing it and that is a problem. If you are not going with the project you will be out of a job sooner or later. Then someone will take over that will make the decision you couldn’t do.

    This is a race to the bottom. Morals, integrity, honesty, responsibility and foresight are only obstacles in this logic (because the competition is not bound by them which gains them an advantage).

    It’s simply cheaper now to build everything in the car always and run an operating system that manages all these things and can control what you are doing in your car.

    Cory Doctorow held a great keynote about this some ~10-ish years (?) ago with the title “The coming war on general computation” where he explained the side effects of putting DRM in every stupid appliance. The side effect here is that we cannot hack our cars to switch on the heated seats (or whatever other feature BMW is not allowing us to use for free) because of DRM. It is not “our” car, even though we bought it.


  • I agree, but as long as we still have capitalism I support measures that at least slow down the destructiveness of capitalism. AI is like a new powertool in capitalism’s arsenal to dismantle our humanity. Sure we can use it for cool things as well. But right now it’s used mostly to automate stuff that makes us human - art, music and so on. Not useful stuff like loading the dishwasher for me. More like writing a letter for me to invite my friends to my birthday. Very cool. But maybe the work I put in doing this myself is making my friends feel appreciated?

    Edit: It’s also nice to at least have an app that takes this maximalist approach. Then people can choose. If they’re half-assing it there will be more and more ai-features creeping in over time. One compromise after the next until it’s like all the other apps. It’s also important to have such a maximalist stand in order to gauge the scale in a way.




  • It’s also a great example why these mega corps should be broken up into smaller pieces.

    If forced arbitration persists (and this argumentation from Disney is successful and then used as precedence) any service used from one company can be used to forever ban you from taking legal action against that company again even if the service and the reason for the legal action have nothing to do with each other.

    Am I right in understanding that this case is about someone dying from eating in a Disney owned restaurant that by accident was a Disney+ subscriber?

    If one company owns everything like Amazon, Google, Apple and in the future maybe even water supply, garbage collection, operates my car and is my insurer or bank account (and owner of one of the 4 remaining fast food chains in the country) how can people actually sue a company then ?


  • Well, but general relativity teaches us that all coordinate systems (also constantly moving, but not accelerating ones) are equally relevant. This means that the one with earth as it’s origin is as correct as one where with the center of the galaxy (or the sun ) as is every other. So the one where earth moves somehow through space is just as random as any other.





  • I have no idea how to fix the problem, but I’ve read somewhere that burn (a relatively new machine learning framework in Rust) is capable of loading models like stable diffusion. As Burn is built with webGPU and all the shader transpiler-stuff that comes with it doesn’t that mean that it can also run easily on (even older) AMD cards? I think what’s lacking is equal performance as nvidia drivers are heavily optimized already.

    Maybe someone knows more here?



  • IIRC most successful VCs invest very early and get out often early-ish too. The real enshittification that dangers the actual position of the company often happen much later. At that point the company is traded publicly and there’s a large anonymous body of shareholders - they only care about profits. VCs are actually a little smarter and care about longer time frames as in that early stage often much larger (relative) growth rates are possible.

    At a late stage (think Google, Twitter, Facebook, Reddit etc today) growth is much more difficult. How could Google grow today? They’ve saturated the search market years ago. So the only way of making more money is by sucking more money out of their existing user base. And they absolutely need to do it, as there’s huge pressure on the managerial class to do it, because the shareholders demand it. If the managerial class doesn’t do this (because often some older idealistic people know it would compromise the quality of the product), or they aren’t capable of doing it - they will get replaced by people who are more willing or capable - even if it’s detrimental for the company when viewed longer-term. VCs i would argue care all about profits, “but”. (they are smart enough to see the big picture. They are also small enough or “few enough” that they can communicate among themselves in order to agree on a more wise plan. That’s why they often get out once most of the possible (easy) growth has been achieved. They either know that now growth is much more difficult, or that the company’s value is much more stagnant - ow might decrease even. They can get out and invest their money in other more promising endeavours.

    The shareholders of large publicly traded companies are not that coordinated as they cannot really agree on anything other than just “growth”. More sophisticated strategies would have to be negotiated (and communicated) among thousands. The only unifying bond among shareholders is that they want profits. Think about it: many shareholders often don’t even know what companies they own as they are often part of other investment packages. Maybe you’re retirement plan has invested in stocks of 50 different companies, or 10 different fonds that have invested in others still. That is a form of dilution (?). It’s very difficult to communicate any strategy more sophisticated than “profits”. (a side effect is also that many people have invested indirectly or wothout knowing in endeavours that make their life more shitty/expensive when they retire - without knowing it.) There isn’t enough nuance in the wants of the masses as to want any more sophisticated strategy than simply “growth”. That’s why only short term growth can be thought.

    Of course sometimes also large companies can grow 2.5x or something like that. But it’s rare and takes more time. The exception makes the rule here. Early stage growth that VCs bank on is much more explosive i think. More like 10x or 100x.

    EDIT: sorry i typed this on mobile and it shows.






  • Oh, i have a brilliant one:

    A few years ago i spent a lot of time converting .flac-files into .ogg-files in order to put on my oldschool iPod. As I did a lot of repetitive typing - entering $dir / for file in flac ; do convert etc / mkdir -p $somewhere/$artist/$album / mv $somewhere/.ogg->$new_dir/ and so on - I thought: “hm lets just write a loop over loops for all the artists here and then all the albums and at the same time create the nested directories somewhere else… hm actually in the home directory… and later love everything on the iPod at once.”

    so i was in my music folder with the artists-folders i wanted to convert. i did something wrong

    So i did my complicated script directly in the shell. I made something wrong and instead of creating a folder “~/artist/album” I created 3 folders in my current working directory: “~”, “artist” and “album”. hmph dammit gotta try again… but first : i have to clean up these useless folders in the current dir. so i type of course this: "$ rm -r ~ artist album " after about 5 seconds of wondering why it took so long i realized my error. o_O I stopped the running command, but it was (of course) too late and i bricked my current installation. All the half-deleted config files made or impossible to start normally and extremely tedious to repair it by hand, so i reinstalled.