Another Tale of Two Link Aggregators

One of my blog posts, the one about .git crawlers, got posted to Lobste.rs, a computers and computer science oriented link aggregator, and to Hacker News, a startup, venture capital ideology and occasional science/math link aggregator.

I’ve had another post make it to all three of Hacker News, Slashdot and Lobste.rs before, but on that occasion I compared Hacker News to Slashdot humans. Here, I compare Hacker News to Lobste.rs humans.

I put the post in question on my web site just after 2026-07-05T20:33:27Z.

The Lobste.rs post appeared 2026-07-06T13:53:15Z.

The Hacker News post appeared 2026-07-06T23:10:32Z.

That’s a cut at visualizing how traffic rose after each link appeared, by counting retrievals of https://bruceediger.com/posts/git-config-spider/ in every wall clock hour. It’s pretty obvious in which hour of the day the Lobste.rs post showed up, and when the Hacker News post appeared.

I didn’t post the link to either Lobste.rs or Hacker News, someone else did. I’m not that into self-promotion. I do think that people watch both sites and “arbitrage” links between them so as to gain karma or clout for themselves.

Criteria for human retrievals

I don’t think that just counting a post’s URL in Apache “combined” format log files is an adequate measure. Many bots, RSS readers and griefers like OpenAI and Anthropic retrieve every URL on my web site many times a day. So exactly as before, I’m using this standard to determine which requests to count:

An IP address that:

1. Retrieves /posts/git-config-spider with lobste.rs or news.ycombinator.com referrer
2. Also pulls /css/style.css with bruceediger.com referrer
   • maybe should be same or later timestamp than (1), but this is harder to enforce.

counts as a “human” retrieval of a URL. I am probably under counting humans with this.

I used the combined log file format tool to find log file entries that have two things that match:

combined -f ipaddr -e 'url~/^\/posts\/git-config-spider\// && referrer~/lobste\.rs/'

That invocation prints IP addresses of requests that ask for /posts/git-config-spider/ and have a Lobste.rs referral. I believe I could have used grep or grep -E to do the same sort of thing, but my tool seemed cognitively easier and gave me more faith in the result.

Ordinary shell scripting and gnuplot did the rest for me.

Results

	Upvotes	Comments	Human IP addresses	Other posts viewed
Lobste.rs	23	2	39	5
Hacker News	17	2	465	23

One Lobste.rs comment was helpful to me, describing what the .git/config spider might be looking for. The other Lobste.rs comment reinforced that web crawlers are indeed looking for .git/config files everywhere. Contrary to reputation, the Hacker News comments were not useful.

At first, I was surprised by how incurious the Lobste.rs humans were. On a per IP address basis, 5/39 = 0.128, 23/465 = 0.049, which indicates that Lobste.rs humans are in fact more curious than HN humans.

As far as user agents go, Lobste.rs users are all on their cell phones, and virtually all use Android, except for one iPhone user, one Windows desktop user, and one weirdo using ELinks on Linux. I end up using lynx once a year or so, but now I’m going to try Elinks. Thanks, anonymous Linux weirdo!

	Hacker News	Lobste.rs
Total	481	39
Android	122	36
iPhone	111	1
Windows	76	1
Mac OS	220	1
Desktop Linux	60	1
Firefox	168	36
Chrome	8	0
Safari	304	0
ELinks	0	1

The numbers above don’t quite add up. For instance, 481 total unique Hacker News user agents, versus 465 human IP addresses. I do think they either (A) help point out qualitative differences, or (B) constitute a lesson in small sample size.

The nearly universal use of Firefox on Android for Lobste.rs seemed wrong, but manual inspection revealed a variety of Android versions (10 through 17), and a variety of Firefox versions (128, 140, 151, 152, 154), so I’m forced to believe it.

I do not believe the Hacker News Safari count. Something’s not right there, but I’ll be dipped if I can figure it out.

Side effects

The Lobste.rs web site appears to have made a test retrieval at 2026-07-06T12:52:54Z from 68.183.100.95, the IPv4 address currently in DNS as the A record for lobste.rs 68.183.100.95 made two requests, 21 seconds apart. The user agent string for the first request was lobste.rs for 186.19.126.53. 186.19.126.53 is the A record for cpe-186-19-126-53.telecentro-reversos.com.ar. That’s good, because whois 186.19.126.53 indicates that address belongs to “Telecentro S.A. - Clientes Residenciales”. Is this the IP address of the person who submitted the link to my post, or something else?

The second GET has a referrer of https://lobste.rs/s/heyyj9, which is the Lobste.rs post linking to my blog post. This is an hour earlier than the timestamp on the Lobste.rs post. Does Lobste.rs have a “cooling off” interval, or a wait for moderation?

It’s well known that Hacker News has a large number of automated “users” that eagerly look for new links, like an instance of Drakma which has requested every one of my blog posts that get linked to by Hacker News within seconds of them getting posted. It looks like Lobste.rs has its own compliment of automated hangers-on, including at least one Slack instance. One weird GET had a user agent of

OpenClaw daily digest (+https://news.ycombinator.com/; +https://lobste.rs/)     

That seems to imply someone is collecting posts linked-to by HN and Lobste.rs for “AI” ingestion.