Apartment Finder
2024
A Selenium scraper that watches apartment listing sites and surfaces only the listings matching my own filters. Written because I was refreshing the same pages by hand several times a day and losing the good listings anyway.
- Python
- Selenium
- Web scraping
- Automation
The problem
Apartment hunting in Gothenburg is a queue-and-refresh game. The listings that fit my criteria were a small fraction of what got posted, and they did not stay open long, so the cost of checking manually was high and the hit rate was low.
How it works
- Selenium drives a headless browser through the listing sites, because the pages render their results client-side — a plain HTTP fetch returns markup with no listings in it.
- Each listing is parsed into a small record: price, size, area, date posted, link.
- Records are filtered against my own criteria, so what comes out is a shortlist rather than a feed.
- Seen listings are remembered between runs, so a scheduled run only reports what is new.
What I learned
Scrapers are easy to write and annoying to keep alive. Selectors break whenever a site ships a redesign, so the parsing layer is the part worth keeping small and isolated — everything else survives a change to the page.
It also made the case for deduplication state early. Without it a scheduled scraper is a machine for re-sending you the same listings until you stop reading its output.