Ollama, openwebui, running the ~20GB Qwen models on a 3090

Absolutely

So much important government computing happens on Linux. Almost every server and supercomputer is on some form of Linux.

Fuck musk and all that, but this was a damn cool stunt and some seriously impressive engineering.

A little bit of airflow can make a huge difference. If it works it works. I think the doubters have just never experimented.

You don’t need to check that the machine code your compiler made matched the code you wrote. You don’t need to check that the commit message an editor sent matched what you typed.

You absolutely do need to check and correct everything that comes out of AI because it’s at best like 80% correct, and that’s if the subject matter is easy.

People said this a year ago and I tested it then and it wasn’t great. I’ll believe it’s better but it’s hard to believe it’s “fixed”. I’m sure I’ll get a chance to try it again sometime. No way I’m paying for it myself lol.

Firefox and Safari both have insecure website warnings, don’t know why they have X’s there

Orange is right … in that this is not how statistics works

I went and looked up that headline. It’s predicting solar system decay one billion years after the sun transitions to a white dwarf

installing an app Apple hasn’t signed requires the command line, for example

It doesn’t, but it does make a scary popup, and requires the user to know to go to settings to find the section to override that popup.

Yeah its definitely useful for topics you personally don’t have much experience in. It’s much less useful when trying to use it for topics you personally are an expert in.

This is basically the same as asking someone who knows a bit about everything to recite obscure information from memory. I doubt most humans would do better.

Sure, but a good web resource or even a good reference book would provide better help faster. Unfortunately it’s getting harder to harder to find those good web resources as search results get overtaken by AI slop.

iterate until the result is correct, then give me a one-page document explaining what I need to know, with cited evidence — far faster than I’d do it

I have gotten it to do this in certain situations, like the other day I wanted to simplify a math formula, so I gave it a loop with a Python script that checked its solution against the original reference version. This worked pretty well.

But I had to write code specifically for that situation. Even with a good skeleton to start with, it’s a non-negligible amount of work to get that set up. I feel like the scenario in which this is useful is kinda narrow: when I have a very good idea of exactly what I want, but some step along the way is a hassle. General software engineering, like making a whole app, is far too open ended, and most of the sub-problems I encounter in software engineering seem either too open ended or too small to benefit from this approach.

That also doesn’t account for the speed of models. My experience is a ~20GB locally hosted model takes like 1-5 minutes to produce a good length response, and the few times I have used online models they are often slower. A few minutes per iteration, accounting for debugging when it goes off track, is not exactly fast or hassle free.

I just feel like the trade off where using AI vs doing it all myself is pretty limited in when AI offers an advantage.

Eh idk I still have it make mistakes constantly.

Recent example: I asked AI to help me come up with a search/replace string for editing a file in vim. My prompt was something like “I am editing a text file in vim and I need to replace <description of pattern> with <description of pattern>. How can I do this with :s/ ?” And it spat out something that didn’t work. It was not valid syntax (for vim, I think it was valid standard regex, or close to it), and it didn’t quite follow the pattern I was trying to describe (although ofc that could be my fault to some extent). But it was useful in that it pointed me in the right direction to come up with a correct formula with some follow up googling and experimentation.

That has been very typical of my experiences with AI. Useful sometimes, but absolutely not “does everything to the point you don’t have to think about it” that seems to be a common opinion.

For context I’m using ~20GB local models, so not Claude, which people who pay for LLMs swear by.

You have to scroll half the page just to get past the ai and ads.

Why do I need a slightly smaller, less capable phone to stick on the back of my phone?

My wife enjoys cooking, I don’t really like it.

My wife hates doing dishes, but I find it a nice excuse to listen to a podcast while doing something menial.

It’s good to get comfortable in the command line, including over ssh.

Especially considering ai and documentation along the way?

If I am going to ask AI or Google about something, I just do that to help me find the answer and then apply the answer myself. That way I learn, and can double check the AI isn’t hallucinating, at least on in obvious ways.

As for documentation, I keep a notes folder with detailed notes on manual configuration I’ve done, how and why, and the things I’ve learned along the way. I’ve found it’s both useful to help remember the things I’ve learned, and it is useful to go back to refer to.

One day, in the name of organizing, my mom put all of our game CDs into a a compact container and threw the original boxes away.

Yeah that was a sad day…

free cooling courtesy of the icy death vacuum of space

Cooling in space is actually really hard. The only way to get rid of heat out there is to radiate it away as black body radiation, which is a really really slow process. It’s like wrapping your computer in a really really thick blanket.

The solar power is the only thing space has going for it.