• JackGreenEarth@lemm.ee
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    6
    ·
    6 months ago

    Can you search the screenshots with OCR though? That’s Recall’s main selling point

    • Aux@lemmy.world
      link
      fedilink
      arrow-up
      59
      ·
      6 months ago

      You can start by running sudo apt install tesseract-ocr and then reading its docs.

      • MacN'Cheezus@lemmy.today
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        6 months ago

        It appears to be as simple as tesseract <infile> <outfile>. Possibly could even pipe (or tee) the screenshot straight into that and save both an image and a text file in a single command line.

        So something like this should do the trick:

        gnome-screenshot -f - | tee /Microsoft/yourPrivacy/$(date +%s).png | tesseract - /Microsoft/yourPrivacy/$(date +%s).txt
        

        Skip the database, just use grep to search that directory if you need to find anything. Voilà, homemade Recall.

        • Aux@lemmy.world
          link
          fedilink
          arrow-up
          1
          ·
          6 months ago

          It is much better to search using ElasticSearch or Sphinx. Grep is super slow, non indexed and can’t do natural language full text searches. It’s pretty much useless for any real world text search you’d want from OCRed content. And all these better tools are free and open source, so really a no brainer.

          • MacN'Cheezus@lemmy.today
            link
            fedilink
            English
            arrow-up
            1
            ·
            6 months ago

            I’m sure there are many ways to improve on this solution, but they would all require significantly more effort (ElasticSearch isn’t exactly trivial to set up).

            This is really just a proof of concept, the most minimal viable implementation that gets you something similar in terms of functionality.

            For instance, Windows Recall stores OCR content tagged by app, this solution doesn’t. Also, as others have mentioned, a practical implementation should likely check if anything has changed at all and discard any screenshots that don’t have any new data.

      • not_amm@lemmy.ml
        link
        fedilink
        English
        arrow-up
        9
        ·
        6 months ago

        I found a small command to run KDE Spectacle (screenshot software) with Tesseract so I can OCR a screenshot if I want to, I only had to install Tesseract and a main language, you could easily do the same with an API and/or a local AI.

      • MacN'Cheezus@lemmy.today
        link
        fedilink
        English
        arrow-up
        3
        ·
        6 months ago

        Llava and Bakllava are two Ollama models than can not only extract text but also describe what’s happening on screen.

        Using tesseract-ocr, as the other guy suggested, is probably simpler and less resource intensive though.