• 1 Post
  • 4 Comments
Joined 1 year ago
cake
Cake day: June 16th, 2023

help-circle



  • In practice it’s not so easy without some manual curation. News sites post a lot of filler stuff and you don’t want to start spamming yourself with every article posted to <insert magazine here>. Even on higher-traffic subs you don’t generally see more than one or two posts from the same site on a given day. It’s generally more effective with something repeatable and reliable like a weekly column where the expected “quality” is invariate. Certainly you can front-load the manual curation by building a set of filters into your scraper, but whether you filter the results at the front or the end of the pipe, you still need some kind of heuristic for what constitutes “good” content, and that’s frequently a moving target.