Rendered at 15:40:12 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
kgeist 14 hours ago [-]
They use WAL in SQLite. If I continuously perform reads/writes so that they overlap with no gaps, I can make their VM go down because SQLite will not have time to initiate a checkpoint to trim the WAL file. SQLite waits for a time window without any active reads/writes before starting a WAL checkpoint. If there isn't one, the WAL will grow indefinitely, eating up all the disk space on the VM.
It's in SQLite's documentation, and almost no one switching to Sqlite seems to be aware of it because no one discusses it in blog posts like these. I guess most projects switching to SQLite have very low traffic and no malicious users (yet)
mikae1 2 hours ago [-]
> I guess most projects switching to SQLite have very low traffic and no malicious users (yet)
Lobsters is invite only. That helps. :)
abetusk 14 hours ago [-]
Could you go into more detail about this issue and provide some links to the documentation?
I'm absolutely no expert, I'm just reading about it now, but from the SQLite WAL documentation [0]:
> 7. There is the extra operation of checkpointing which, though automatic by default, is still something that application developers need to be mindful of.
and from their "checkpointing" documentation [1]:
> By default, SQLite does a checkpoint automatically when the WAL file reaches a threshold size of 1000 pages.
I'm only skimming but I see no mention of waiting for an idle time window.
>However, if a database has many concurrent overlapping readers and there is always at least one active reader, then no checkpoints will be able to complete and hence the WAL file will grow without bound.
>This scenario can be avoided by ensuring that there are "reader gaps": times when no processes are reading from the database and that checkpoints are attempted during those times.
Dunno, maybe Rails has a built-in workaround for this.
My workaround was to run a separate thread that monitored the WAL size on disk every second. If it went above the target size of 8 MB, my framework would enter "slow down" mode, where all reads and writes were artificially delayed by calling "sleep()", starting at 16 ms and gradually increasing the sleep time based on a few heuristics.
This allowed the application to have short gaps with no reads or writes, so the checkpointer could actually proceed.
abetusk 14 hours ago [-]
Thanks, this is really embedded in the documentation and is not at all even hinted at in the above documentation.
This seems like a huge DoS security hole.
I encourage you to write a blog post about this. It seems especially relevant.
inigyou 14 hours ago [-]
Why not grab a rwlock so that no readers or writers can proceed and all existing ones are drained, and then force a checkpoint? What's with the exponentially increasing sleep?
Seems to indicate a collection cron with a manual query to truncate can fix it. This is fun little things I like to learn. SRE Easter eggs.
gregoryl 3 hours ago [-]
Those SRE people live on excitement, its basically a gift ;)
shirol 3 hours ago [-]
That's only true in theory. In practice, you'll be rate-limited first. Even if you aren't, the database won't be the bottleneck because SQLite is so fast. Their Ruby on Rails stack will suffer thread starvation way before the WAL grows. Rapid requests still leave microsecond gaps between context switches for SQLite to run an automatic checkpoint. To actually make SQLite stall, you'd need an endpoint that lets you hold a massive write transaction open indefinitely, which lobste.rs doesn't expose anyway (I hope).
badatnames 14 hours ago [-]
Gippity claims this is only a problem with long-lived readers, the same is true in many other DBMS.
kgeist 14 hours ago [-]
I reproduced it in Sqlite with short-lived reads/writes though. Other DBMSes seem to not have this issue (IIRC MySQL will block a write if WAL falls behind)
homebrewer 18 hours ago [-]
It's been fairly unstable recently, pages sometimes render for several seconds which I've never seen under MariaDB. Used to be instantaneous, always.
Sometimes (maybe 5% or less) the request won't render at all, and you get a browser error page.
Today they ran into this bug, lost a bunch of voting data, and went into read-only mode for several hours:
I wonder how much of this is usual bugs which crop up during major database migrations, and how much is caused by choice of SQLite.
eterm 17 hours ago [-]
SQLite definitely seems like a poor choice for dealing with many concurrent requests.
Maybe it's improved since I last used it, but to my knowledge SQLite essentially forces all writes to be serialised, at risk of data corruption otherwise.
There are tricks for improving the performance such as WALs, but that is merely a performance boost rather than genuine concurrency with things like row-level locks that you might find in other databases.
I guess if the whole thing is architected with a write-through cache that handles concurrent writes and deals with serialising all the writes, then it can be a single writer streaming changes through to the database, but then you still have a point of serialisation, it just will manifest itself slightly differently.
And SQLite is something that will give you constraints you will always have to consider.
Whereas running mariadb or postgres on the same machine would deliver similar benefits without a risky migration.
If your DB is small enough to run as a SQLite database, then it probably ought to have never been on a different machine in the first place.
There is a very happy medium between SQLite and a database on a different machine, one I am continually surprised to see people ignore.
wild_egg 15 hours ago [-]
> SQLite definitely seems like a poor choice for dealing with many concurrent requests
Can you qualify "many"? SQLite easily handles 100k+ writes per second and it's not hard to have app layer code serialize and batch writes to take advantage of that speed. Concurrent writes require a ton of overhead and your system and code can be quite a lot faster and simpler if you just skip the idea of them altogether.
inigyou 14 hours ago [-]
SQLite serialises all writes - only one can proceed at a time and others must wait until that one is finished
victorbjorklund 6 hours ago [-]
Writes are fast unless you are doing some complex transaction stuff.
inigyou 5 hours ago [-]
Each one includes a few fsyncs
torginus 8 hours ago [-]
Can't you do infinity read replicas? I don't think the write load on lobster.rs is severe - maybe 1 comment/sec, with an acceptable replication delay of 60 sec?
If you can do that, you can have infinity instances.
eterm 5 hours ago [-]
With WAL, you don't need read replicas, you can have lots of readers and they don't get blocked by writers.
As an aside, on replication, eventual consistency is not a panacea:
If I make this comment, it's absolutely fine if you don't see it for a minute.
If I make this comment, it's absolutely broken if I then don't see it on my subsequent requests. I'll think the site has broken, and try to resubmit my comment.
You can relatively easily work around that one by pinning people to instances, but that's still yet another thing to consider, and those kinds of considerations add up when you're dealing with distributed systems.
However, this a single instance, it's way too early to talk about replication.
Given that just hitting their front-page is taking 6+ seconds, they've got a performance problem that needs fixing, and my hunch is that they have some kind of "last accessed" database entry, which turns every request into a write.
veqq 16 hours ago [-]
edit: misread requests as writes
eterm 15 hours ago [-]
I wouldn't expect that problem to be solved at the database layer, that's what http caches are for, then application level in-memory caches, then finally if neither of those hit, go to the database.
For example, one of the biggest optimisations that Hacker news does is that it serves logged out users from a cached copy of the front-page.
Logged out users don't care/notice about comment counts, they don't notice that it doesn't update as often, they can't be hiding articles so you can serve the same front-page to the millions of anonymous users and bots, and update that cached copy once and on a slower cadence than every request.
15 hours ago [-]
alfiedotwtf 14 hours ago [-]
Depends on the architecture..
They could have one SQLite instance per user and then have a single sweeper that goes through all last writes and then replicates it to the main instance - eventual consistency fanned out across files
anonzzzies 16 hours ago [-]
Not sure if this is SQLite or driver/Rails issues? We use it for sites and systems with a lot more (but similar) traffic and have no issues; we did write the drivers we use ourselves (for Node) as we got into much issues with the popular/existing ones.
srcreigh 18 hours ago [-]
Pages (especially threads while logged in) taking seconds to load definitely happened to me pre SQLite migration.
abrookewood 16 hours ago [-]
I jsut had this happen - it was my first visit to the site and probably my second page load. Tried opening the About page and it was completely blank. Thought it was odd & reloaded, which rendered correctly.
ekjhgkejhgk 8 hours ago [-]
It's super slow.
pudgywalsh 17 hours ago [-]
It must be absolute torture for all 14 of their users.
jdiff 17 hours ago [-]
Yes, a pox on othersite, all hail currentsite.
small_scombrus 9 hours ago [-]
Currentsite isn't as good as it was in pastdate, the grass is greener on othersite!
CursedSilicon 16 hours ago [-]
Why do we need any other sites at all? Let's just all use Facebook
evanelias 18 hours ago [-]
Some of the cited reasons for moving off MariaDB [1] seem misguided, in my opinion. Especially the part about "K1 are very enterprise-focused, so the database is likely to focus its work on features that are not relevant to us. There's increased risk they drop the free/open source version we use"
K1 acquired the commercial entity behind MariaDB Enterprise, but that's separate from the non-profit MariaDB Foundation. And there's literally zero risk of the MariaDB server suddenly going closed-source; as a fork of MySQL (which is GPL), this is not even legally possible!
The GPLv2 doesn't prevent a company from distributing binaries and making the source available on request while development is done behind closed doors. This wouldn't really be "open source" as we typically understand it and might be cause to exclude it from some Linux distros, for example. I'm not saying that K1 would do this (I have no idea who they are or what they do), but I also think it's a reasonable risk to consider.
inigyou 16 hours ago [-]
That was precisely how the original free software movement started, actually. Work was done behind closed doors and if you didn't like it, you released your own version with your changes.
evanelias 17 hours ago [-]
I don't see how that would be a realistic risk? Any MariaDB user (including Lobste.rs) could request the source, and per the GPL it must be supplied, so this isn't an effective strategy for any entity to go closed-source.
And in any case, https://github.com/MariaDB/ links to mariadb.org which is run by the MariaDB Foundation, not the commercial enterprise owned by K1.
inigyou 14 hours ago [-]
Even with that in mind why switch to sqlite rather than PostgreSQL, which is roughly equivalent to MariaDB?
zinodaur 16 hours ago [-]
As a big user of MySQL - moving off of it does seem like a good idea, if you can. The longer you wait, the harder it will be to flee
evanelias 14 hours ago [-]
It's a large ecosystem with multiple server vendors (Oracle MySQL, MariaDB, Percona, VillageSQL) that are relatively easy to migrate between, compared to moving to a completely different DBMS ecosystem.
Sometimes there are valid technical motivations to move away completely, but a nonsensical fear of a massively-used GPL database somehow becoming closed-source (with no surviving forks or continuity) is not one of them in my book!
alfiedotwtf 14 hours ago [-]
Any reason besides trust me bro?
zinodaur 11 hours ago [-]
- Its query planner is ancient and broken - it comes up with very bad plans. Every time we rely on it, we regret it eventually
- you have to use locks to prevent mangled write transactions, instead of the db handling it. Jepsen report is scathing
- its pretty hard to set it up in semi-sync replication mode, a.k.a. "only return from a transaction commit when the transaction is present on at least one other replica". Once you've set it up, you have to deal with insane things like transactions being visible on replicas before leaders, or even worse transactions visible on replicas briefly but never visible on leaders or other replicas
- every knob and feature contains subtle bugs, sometimes ones as scary as index corruption, that you only discover after it has caused you pain
I don't know why you would choose it unless you were already running it
evanelias 49 minutes ago [-]
The query planners have diverged between MySQL and MariaDB; which one are you talking about? Both have received improvements in recent years, so the specific version is relevant as well. And at least both systems have supported index hints for decades (unlike another popular database that only very recently acknowledged their necessity in large production workloads...)
The locking anomalies are a well-known and well-documented side effect of InnoDB REPEATABLE READ differing from the standard in certain pathological cases. The real-world relevance is minor, especially since companies using external write-through caches are already making much worse inherent trade-offs in that area. Regardless, MariaDB fixed it, while providing an option to revert to the old behavior for MySQL compatibility. And fwiw this only comes up because MySQL/Maria defaults to REPEATABLE READ; yet meanwhile the overwhelming majority of Postgres users are totally fine with the much weaker guarantees of READ COMMITTED, because that's their default isolation level.
Semi-sync has some odd behaviors indeed; there are conceptual trade-offs that must be made if you want synchronous-like replication without the full performance penalty! Do you have some novel academic solution for this that you can propose?
Your last bullet is lacking in details so I cannot comment.
ksec 4 hours ago [-]
>I don't know why you would choose it
Vitess. Also I often wonder if these system are on latest mySQL? I have read plenty of mySQL opinions that still based their experience during 5.x era.
edoceo 18 hours ago [-]
And even if it did go closed-souce, your existing install can run for ages because the vendor cannot force upgrade. One of my favorite features of FOSS.
NuclearPM 17 hours ago [-]
How could it go closed source?
c45y 17 hours ago [-]
Probably in the way Minio did.
Technically the source is still available but the controlling entity behind it have shut down all changes
vasco 17 hours ago [-]
This makes little sense but I guess they get little traffic so pretty much any database will work for them.
If traffic grows they will find problems, then limitations, then move to postgres. And if not it's fine. Story old as time.
On my Oracle databases, I run the dbms_stats.gather_database_stats package procedure once a week. SQL Server also has stats for sure, and I would be surprised by a relational database that did not.
SQLite will quietly gather stats with each database connection. Running an explicit analyze devotes full attention to an otherwise piecemeal group effort.
2. I don't think rows are physically removed when deleted, instead they are marked. Removal will definitely happen at vacuum.
3. There is a backup command in the CLI that will make a transaction consistent copy of the database file, and there are lower-level C functions behind it that can be used with object code.
4. It is important to remember the many restrictions of WAL mode, including the impact on ATTACHing multiple database files: ACID consistency is lost.
'Transactions involving multiple attached databases are atomic, assuming that the main database is not ":memory:" and the journal_mode is not WAL.'
cromka 18 hours ago [-]
Ironically, this post resurfaced on HN with my comment here shown as if made 15 minutes ago, which I in reality did last week?
stymaar 18 hours ago [-]
Second chance pool, most likely.
mh- 17 hours ago [-]
I didn't realize that re-timestamped comments. That's a pretty unexpected behavior, if true. I wouldn't like if that happened to a comment I left - a lot can happen in a week.
inigyou 16 hours ago [-]
Every time it happens there's a sub-thread asking about the weird timestamps and dang replies to it saying changing the timestamps is done so that there won't be a sub-thread about the weird timestamps.
tayo42 15 hours ago [-]
I'm pretty sure this comment chain is in every popular 2nd chance pool post though anyway
stymaar 8 hours ago [-]
True. I don't understand why they overwrite the date at all actually, for either the post or the comments, that sounds like the hackiest way of implementing the “second chance pool” feature.
varun_ch 17 hours ago [-]
Does HN still use simple files for storing its data?
dcmatt 13 hours ago [-]
Wow, lobsters seems refreshing! Invite only to contribute must be why they can run on sqlite so effectively. I've run into so many issues with WAL (real time market data) unless I'm painfully deliberate with reads and writes. Don't feel like jumping on the "ohhh invite me" bandwagon, but definitely subscribing to their feed.
wewewedxfgdf 8 hours ago [-]
I would visit lobste.rs more if it filtered out stories that are on Hacker News.
I'm not interested in a rehash of Hacker News but I am interested in more tech news.
Anyhow on topic: I find it puzzling the whole "use sqlite in production!" thing.
- you're stuck on a single machine now.
- you don't have remote database access.
- you have to live with the inflexibility of sqlite's schema change capability.
- all writes go through a single writer.
- fewer data types
- fewer backup/security/high availability options
- foreign keys and strict disabled by default.
- you're single core in a multi core world.
It's not a terrible decision but when you can choose Postgres, why would you go with sqlite?
The site is performing poorly for me. For example, the login page took 9 seconds to load, same with the home page.
dzonga 4 hours ago [-]
my take to benefit from SQLite:
make sure you're not doing a bunch of migrations. that's the SQLite weak point that I know of & have experienced.
use SQLite - as if you're using a KV or better yet maybe a CQRS approach.
cromka 5 days ago [-]
Wait, no edge cache, just one node with SQLite db?
packetlost 18 hours ago [-]
For a relatively low-traffic site like Lobster's, yeah I'm not surprised they don't need anything crazy. SQLite is really efficient and modern hardware is really fast.
booi 15 hours ago [-]
I don't think SQLite is particularly optimized for modern hardware. It's just crazy efficient with a regression test suite many projects wish they could have.
thomasdziedzic 3 days ago [-]
Caddy does cache a portion of the traffic, and what it can't cache makes its way to rails.
hnarn 11 hours ago [-]
Moving from MariaDB to SQLite and not Postgres for a web application of this scale is insane. I wonder what went into the discussion behind this.
amazingamazing 16 hours ago [-]
Software “engineering”. Thank goodness bridges aren’t built with this level of rigor.
In any case I will take an invite if one has one because the community is solid.
Peanuts99 7 hours ago [-]
I've got a few planks over a ditch in my garden. The bridge only needs to be designed for the load and then some. In this case, this seems to work just fine.
abetusk 15 hours ago [-]
Give your email or send me a message (abetusk@mechaelephant.com) so I can send the invite.
amazingamazing 13 hours ago [-]
Done. Just added to profile, thank u!
MitziMoto 11 hours ago [-]
Although the site is very fast for me as of this writing, it looks like they're running into an issue:
I hope they get it sorted soon, I'm rooting for sqlite.
Kuyawa 18 hours ago [-]
No intention to antagonize but... why not postgre? Even my side projects run on postgresql with no overhead at all
dewey 18 hours ago [-]
It's also explained in the post shared:
---
I've heard "why not PostgreSQL?" a few times this week. It was even our original plan in #539! Well, it was a pragmatic choice in two different ways:
The person who volunteered to do the work used SQLite.
I don't want to use solutions that are bigger and more complex than our likely needs. Postgresql is my default for projects, but it does have the added complexity of being a separate service to run, tune, and maintain.
gnulinux 18 hours ago [-]
> run on postgresql with no overhead at all
Citation needed. I love postgres as much as the next person, but it does have more overhead than Sqlite which is in an in-process db linked through compiled C code, it doesn't run as a separate server. Very significant difference that when you use Sqlite db, there is typically no db process other than your application logic, unless you implement the server yourself. If you don't want your application to have multiple processes (say, as a toy example) then it totally makes sense to prefer Sqlite just for this reason. Sqlite and postgres are different tools, they serve different purposes.
christophilus 14 hours ago [-]
If you run it on your one app server, it’s not much more work than SQLite. SQLite requires some special care when you’re dealing with multiple processes, and backups are about the same amount of effort.
adenta 18 hours ago [-]
Where are you hosting postgres
dewey 18 hours ago [-]
On the same machine where they are currently storing the sqlite file maybe? It's not that hard to run. A sqlite backup cron is also not that much more work than a pg_dump cron.
inigyou 16 hours ago [-]
if they have only one server, then on that server. If they have a load-balanced front end and central database, then on the database.
rfmoz 17 hours ago [-]
Another SQLite user here. My site gets constantly hammered by crawlers, and SQLite just keeps going on hit miss. Anyway, the real magic is on the ram and the ssd perf.
cdrini 17 hours ago [-]
> keeps going on hit miss
(Is this a typo?)
arjie 16 hours ago [-]
Pretty cool. I do default to SQLite these days simply because I can just put it on the host, set up a PVC to it, and run the app in Kube pointing at a persistent SQLite. Absent some other need (it's a fork of psql- or mysql-based software) I prefer using it for simplicity.
I do have to say that I've found lobste.rs generally a pretty slow site to render and navigate and I wonder whether that's the Rails backend, or underpowered hardware, or lack of caching. It's more feature-ful than a HN or Reddit though with much lower usage so I wonder what it is.
cenazoic 18 hours ago [-]
May I have an invite, if anyone has one to spare?
abetusk 15 hours ago [-]
Provide an email or email me (abetusk@mechaelephant.com) so I can get yours for the invite.
"Ordinary rsync does not understand SQLite transactions. Rsync will make a copy of ORIGIN into REPLICA, however the copy might not be consistent. Parts of the copy might be from one transaction, while other parts might be from a different transaction. The database copy might be corrupt."
abofh 17 hours ago [-]
I hope it's profitable, because this feels like engineering for engineering sake and not actually improving the product in a way that draws customers.
p2hari 11 hours ago [-]
Everytime Lobste.rs get posted, I get to think should I dive in. This time maybe. Could somebody be kind enough to give me an invite. Email in my profile. Thanks.
abetusk 2 minutes ago [-]
Invite sent. Let me know if you didn't get it.
Levitating 5 hours ago [-]
Lobste.rs seems cool. Would someone invite me?
abetusk 2 minutes ago [-]
Invitation sent. Let me know if you didn't receive it.
luciana1u 17 hours ago [-]
sqlite for a site that debates database migrations is like a chef who eats microwave dinners at home. makes perfect sense and somehow still feels like a betrayal.
blobbers 16 hours ago [-]
Do you feel like lobste.rs is a better version of HackerNews?
veqq 13 hours ago [-]
They're very distinct. Both are good in their own ways like elephants and dolphins. Though HN suffers more under AI spam, it permits wider topics and has a wider userbase.
dmix 15 hours ago [-]
Nice to see Lobsters is made with ruby
DHH has made a big deal about using SQLite in rails
oandrew 17 hours ago [-]
I'd appreciate a lobste.rs invite if anyone has one. Email is in my profile. Thanks!
17 hours ago [-]
uproarchat 16 hours ago [-]
Successfully e-mailed invitation to hn@oandrew.dev.
maxloh 14 hours ago [-]
Hi. Could you send me an invite too? Email in my profile :)
oandrew 12 hours ago [-]
Thank you!
Imustaskforhelp 17 hours ago [-]
> I'd appreciate a lobste.rs invite if anyone has one. Email is in my profile. Thanks!
I'd like one too if someone wishes to invite me as well, Thanks and have a nice day!
uproarchat 16 hours ago [-]
Successfully e-mailed invitation to anonplusplus@proton.me
Imustaskforhelp 16 hours ago [-]
I have signed up, thanks!
You are awesome mate!, have a good day :-D
uproarchat 16 hours ago [-]
Take it easy :)
mixmastamyk 15 hours ago [-]
Please don’t post other’s addresses, after a bit are unable to be removed. If they decide to later it’s too late.
tetris11 5 days ago [-]
Oh wow. Off-topic but quite refreshing to see a site not absolutely plastered in AI posts
grebc 18 hours ago [-]
HN is increasingly bot central, I kind of assume 80% of the respondents are artificial.
Lobsters still got a pulse, largely.
rickcarlino 18 hours ago [-]
Invitation-only is the future of online communities. Lobsters still has a pulse because you need to know someone to get in and it’s a bad look if you invite annoying people.
tverbeure 17 hours ago [-]
I wish lobste.rs would be a tiny bit easier to get access to. Some of my blog posts have been discussed there and every time I try to get access, I can’t get past their IRC stuff.
As in: I just don’t understand how IRC works…
inigyou 16 hours ago [-]
IRC is just Discord but without a hardcoded server address, what's hard about it?
pudgywalsh 17 hours ago [-]
You should overlay a banner on your blog when they send you an HTTP Referer explaining their nonsense. Maybe then they'll get the message.
Then again, maybe they like the ability to critique your posts without you being able to rebut their opinions. It's an effective tactic in online echo chambers.
accelbred 16 hours ago [-]
From their about page: "If you wrote a link that was posted, please reach out in chat, we'd love to have you join the community."
inigyou 14 hours ago [-]
Or redirect to a picture of a testicle in an eggcup.
cpach 17 hours ago [-]
FWIW, you might want to consider publishing your e-mail address in your HN profile or on your website. Otherwise it’s hard for people to get in touch with you.
arjie 16 hours ago [-]
Well, lobste.rs requires an email to invite a user. If you had one on your profile, or on your github, or on your website I would have done it as a drive-by since your site is interesting and pretty lobste.rs-y content.
anukin 12 hours ago [-]
It’s not just irc stuff. There are a lot of elitist mods in lobsters.
devilsdata 17 hours ago [-]
I've been waiting for my lobste.rs invite for years. I joined the IRC for a few days, but being in Australia, rarely saw too much activity. I could chase someone down for an invite, but at that point I just don't care that much.
abetusk 15 hours ago [-]
I was given my invite from a decent person who, I believe, basically has an "open door" policy. I'm happy to pay it forward and give you an invite if you want it. abetusk@mechaelephant.com
Xeronate 17 hours ago [-]
im sorry but maybe connecting to irc is a bit of a filter on the users they expect. especially with ai it should be almost no effort.
cdrini 17 hours ago [-]
> Invitation-only is the future of online communities.
This feels antithetical/dystopian/elitist compared to what online communities are, for me. I think it's ok for some communities to be like this, but I don't think it's the future of online communities.
qsera 17 hours ago [-]
I think it is better to have some elitist communities, than to drag everyone down to some least common denomination...
I think that was one of the big mistakes/oversight that came with the Internet..
cdrini 16 hours ago [-]
I think some are fine, but if I had to choose one I'd choose open access, so I think it's the more important of the two. And in fact it's because the web is open, "no permission needed" by default that allows elitist groups to exist on the web. So I don't think I see it as a mistake. The alternative would be... You can only create a website if you're invited to the internet by someone who already has access? You have to get approval for your website from some authority before it goes public? Those were all potential futures we imo thankfully and carefully avoided!
qsera 6 hours ago [-]
>Those were all potential futures we imo thankfully and carefully avoided!
I am not so thankful. But it is coming from my view of the current world.
I think open Internet for business is good. But open Internet for communication was bad. I cannot think of one good thing that have resulted from an open internet for communication/discussions. It has turned out to result in a collective, perpetual state of seizure for all humanity.
I am someone who had to learn stuff from random encounters with books in the 90s. Even then, I would not have it any other way...
inigyou 14 hours ago [-]
That's actually how it already works, but there are large commercial access providers known as "ISPs". Before there were ISPs, you literally could only get on the internet with a direct connection to someone on the internet, usually a large university.
inigyou 16 hours ago [-]
This is called the Eternal September problem, after the influx of fresh users to Usenet at the start of every school year, being extended indefinitely as AOL brought new users continually to Usenet.
tayo42 15 hours ago [-]
Isn't this how alot of discord works? maybe the communities are leniant with inviting but it still happens. A group chat is basically invite only too and that's pretty popular for communication.
veqq 13 hours ago [-]
Elitism and hierarchy are good. Drowning everything in slop and darkness is bad. Giving everyone something to strive for is good.
Barrin92 16 hours ago [-]
I think we need to remove dystopian and elitist from the dictionary because at this point you can't breathe without being accused of it.
Deciding who is in or out is the fundamental basis of any community, that's freedom of association. That's what defines a community. A place anyone can enter is a train station bathroom and that's what the logical terminus of the open internet seems to be.
cdrini 16 hours ago [-]
I think scopes/criteria/boundaries on communities are fine, I disagree that invite-only (as one implementation of community boundaries) is the future of online communities.
Although the statement "invite-only only is the future of online communities" isn't super well defined (what does it mean to "be the future"?), so I think the comments are debating past each other.
OutOfHere 17 hours ago [-]
That's nonsense wrt invitation-only being the future. I can equally say that free speech is the future. For comparison, Reddit does very well without being invitation only.
inigyou 14 hours ago [-]
Reddit is literally the dead internet right now.
It's also not open access. You have to run an invisible tightrope so its algorithms won't automatically shadowban you and then you have to run a barely visible tightrope to get enough karma to not have your posts automatically removed from every subreddit.
inigyou 4 hours ago [-]
Also, Reddit now requires you to verify your ID with Persona.
Xeronate 17 hours ago [-]
reddit does okay from a profit standpoint. reddit does terribly from a bot or even a good community standpoint.
OutOfHere 17 hours ago [-]
That's an unsubstantiated and false accusation wrt the 80%. If you believe it, why are you even here? Are you one of them?
synqvest 16 hours ago [-]
[dead]
inigyou 14 hours ago [-]
HN has always been "YC bootlickers only"
Aurornis 18 hours ago [-]
Lobste.rs is very anti-AI. They force any LLM topic to be tagged with “vibecoding” even when the majority of LLM posts are not about vibecoding.
AI posts are usually the most commented on. Linus Torvalds’ comments saying that LLMs are actually useful is still on the front page, tagged as vibecoding, and has a lot of comments from people mostly disagreeing with him.
Lobste.rs is more of a monoculture than Hacker News. If you get downvoted enough they stick a banner on the top of the page inviting you to delete your account. It has a nasty side effect of driving away good contributors who don’t align perfectly with the hive mind. It’s a site where you learn to keep your mouth shut if your input doesn’t agree with what the core users want to hear.
arjie 16 hours ago [-]
I originally went to lobste.rs because I wanted to avoid repetitive discussion and get more technical stuff. In time, though, lobste.rs got fairly boring since it's always the same stuff over and over again. I found an alternative means[0] of filtering out boring things here on HN and it's enjoyable to use again.
0: deleting comments of people who aren't interesting from my view and highlighting ones who often say things that are interesting, the user I'm replying to is one of the latter
Aurornis 15 hours ago [-]
> In time, though, lobste.rs got fairly boring since it's always the same stuff over and over again.
There are some interesting comments there from some people who don’t spend time on other sites, but your assessment is similar to mine
My comment about being anti-AI above may not have been clear. They actually talk about AI a lot and AI threads attract a high amount of activity. The difference is that insightful or honest comments don’t have much of a chance because there’s a clear popular opinion on the site and any information that doesn’t fit isn’t welcome.
The giant soupy mess of opinions on HN can be tiresome, but it’s nice to get a wider range. HN is its own little bubble but at least there’s some diversity of experiences to be found.
binkHN 14 hours ago [-]
lobste.rs, quite literally, allows you to filter the entire site by tags, so you only see what you want to see. I wish more sites did this.
joemi 17 hours ago [-]
It's a lot smaller of a community than HN, so it will naturally have a smaller range of opinions merely due to the fact that it has a lot fewer users.
That said, I don't believe it's as much of a hivemind as you've portrayed. I've seen dissenting opinions voiced there which weren't downvoted (or at least not noticeably), while the same opinion in a similar thread here on HN would be downvoted and flagged. So I don't think that getting downvoted enough so that you're invited to delete your account is a common occurrence with productive members.
flomo 17 hours ago [-]
It's fine not to be an AI user, just some people there seem to be holding their breath and stomping their feet until it goes away. Which leads to a really repetitive and boring discussion. (I'm not a member there, but they often have better technical links vs HN.)
maxloh 14 hours ago [-]
Hi. Could you send me an invite? Email in my profile :)
OutOfHere 17 hours ago [-]
With regard to its small size, that's not why. They have for a long while been actively and intentionally hostile to AI.
CursedSilicon 16 hours ago [-]
I love it already!
moron4hire 16 hours ago [-]
Nice. I'll go check it out. Thanks.
Aurornis 17 hours ago [-]
> So I don't think that getting downvoted enough so that you're invited to delete your account is a common occurrence with productive members.
It is! It’s a rite of passage on the site if you dare comment anything that someone might disagree with.
There have been several high profile cases of valued members getting fed up with it and leaving the site. One example where even the community was starting to question if this is really what they want: https://lobste.rs/s/zp4ofg
hagbard_c 17 hours ago [-]
I don't think it is a much a hive mind but it is - or at least was - susceptible to manipulation by individuals who know how to game the system to push their own narrative. I got into a discussion with one of these, a 'furry' who posted semi-relevant articles laced with furry-themed illustrations where those illustrations seemed to be the main point of the articles. After a number of such articles I remarked on this and was banned shortly after, somewhat out of the blue. The moderator did not like this type of 'sensitive' discussion which more or less meant the site was fair game for those willing to abuse it knowing that any discussion on their abuse would be suppressed. Maybe things have changed, maybe they haven't but it did leave me rather disillusioned with the way the place was moderated and as such I haven't been there for years.
BoingBoomTschak 15 hours ago [-]
Being anti-AI is fine by me, but the hardcore Tumblr dogma can be very tiring sometimes; even if only as a reader interested in the valuable technical comments.
mort96 17 hours ago [-]
Driving away silicon valley tech bros who spend most of their time boosting AI is actually a prerequisite for building a good community.
anukin 12 hours ago [-]
Instead invite tech bros from India belonging to the so called premier colleges of India and gatekeep seems to be the vibe of lobsters.
xtracto 18 hours ago [-]
I dont remember why I was kind of barred from lobsters , but I applaud all this behavior you mentioned. In the end, that's how normal clickes of friends/interest groups form (or formed 30 years ago in real life.)
If you got into my group of death metal people, and suddenly started talking about Oasis... we would invite you to get out haha.
Same with patriots.win . They may be a group of right wing biggots, but hey, they did their platform and are happy talking there.
Aurornis 17 hours ago [-]
> I dont remember why I was kind of barred from lobsters , but I applaud all this behavior you mentioned.
Some serious cognitive dissonance here.
I think a lot of people like this idea of a close-knit clique of like minded people, but the communities tend to pick up toxic qualities where membership becomes a game in itself. Anyone who doesn’t fall in line with the established in-crowd gets pushed out via the Byzantine rules that the in-crowd uses to maintain their position.
I still visit and participate, but I’m realistic about what it is. There is some value there, but you have to navigate it like a small subreddit where the moderator keeps tight control over what can be discussed and indirectly or directly influences the allowable opinions on the site. I really liked their original ideas about having an open moderation log and clear rules, but I’ve seen more than enough questionable moderation decisions and selective application of the rules to accept that it’s not as pure as I thought it was going to be.
cdrini 17 hours ago [-]
Hmmmm interesting analogy! I'm not too sure if cliques apply as much to online communities. Cliques are usually exclusionary not because people don't meet the criteria/scope of the clique, but by scale constraints, eg how many people someone can maintain a close relationship with. Online communities are kind of sufficiently different because of this scale. They can grow basically indefinitely, and are only limited by the stated scope/criteria of the community.
OutOfHere 17 hours ago [-]
Lobsters has other problems too. The owner occasionally steals credit from posters, changing the author of posts to his own friends, also taking all the credit for it away. Debating the owner's action results in a ban.
NetOpWibby 17 hours ago [-]
Ironic, considering why Lobsters exists in the first place.
vovavili 17 hours ago [-]
So they basically reinvented reddit.
abc42 17 hours ago [-]
Yeah, it's starting to be a bit idiotic. Some of that stuff is here too.
dgellow 18 hours ago [-]
I honestly don’t find any of this concerning
mh- 17 hours ago [-]
The concern usually only comes when you fall on the "outside" of the prevailing opinion within the group of people voting on comments.
edit: my response isn't about lobste.rs. I don't have an account there.
gspr 18 hours ago [-]
It's a wonderful place. It feels like HN, but with the Silicon Valley attitude dialed way way way down, and with a much narrower scope (politics, economics and entrepreneurship are all off-topic, for example).
NetOpWibby 17 hours ago [-]
The SV attitude isn't there but there's active assholes who wield their moderation weirdly. I deactivated my account before deactivations were added to the logs and I don't remember what made me quit. I've largely avoided Lobsters since.
I joined their IRC channel last week and it's been rather chill.
I do like the invite-only nature of the site though, keeps quality high...while making some folks tyrannical.
cromka 16 hours ago [-]
Every such "club" inevitably attracts a specific sort that badly want to join just for the sense of exclusivity and a chance to virtue-signal their member status. It's same thing with some hard-to-get-into torrent trackers — once in, people become poster-children for Stockholm syndrome and the mods really get off on their perceived power. It's also the same with expensive new sport car models that manufacturers sell to collectors.
My intuition says it's to do with Status Signaling theory .
ai_critic 15 hours ago [-]
Slight clarification: politics that disagree with the zeitgeist of the site and the mods is off-topic.
maxloh 15 hours ago [-]
Hi. Could you send me an invite? Email in my profile :)
christophilus 18 hours ago [-]
Huh. I’ve largely avoided it, but that does sound pretty nice.
dgellow 18 hours ago [-]
[flagged]
dprkh 17 hours ago [-]
If still relevant, you can email me and I will send you an invite. My email is dprkh at fastmail dot com
osinix 17 hours ago [-]
[flagged]
aboardRat4 17 hours ago [-]
[dead]
synqvest 16 hours ago [-]
[dead]
ethagnawl 18 hours ago [-]
[flagged]
veqq 18 hours ago [-]
[flagged]
ethagnawl 18 hours ago [-]
Thank you!
sgt 18 hours ago [-]
What's lobster.rs, some kind of Temu HN? Or is it the real deal?
xgbi 18 hours ago [-]
Way more technical, way smaller.
HN is a lot more wide in terms of audience and subjects, Lobsters is mainly programming.
I love it.
sgt 18 hours ago [-]
I'll give it a shot.
OutOfHere 17 hours ago [-]
It is a closed-minded AI-hating cult.
written-beyond 18 hours ago [-]
It's superior and invite only.
pstuart 18 hours ago [-]
[flagged]
xgbi 18 hours ago [-]
[flagged]
pstuart 18 hours ago [-]
merci!
synqvest 11 hours ago [-]
[flagged]
etbusch 18 hours ago [-]
[flagged]
jdw64 17 hours ago [-]
I think HN is better. Lobste.rs has too much monocultural pressure.
That's not necessarily a bad thing, but honestly, as a foreigner, I have limited English vocabulary, so I end up getting stressed out over being judged as AI. At least here, the admin DEN kindly understood my situation and let me participate after getting permission. But when I look at subreddits and other communities, the stronger the monocultural management, the more fan-like sentiment tends to grow, making it harder for minority voices to speak up.
I'm in favor of LLMs (since being for or against LLMs inevitably depends on regional context—I personally find LLM translation incredibly convenient and helpful), but there are moments when I find it hard to argue against blanket opposition or certain topics. I thought about why that is, and I think it's because Lobste.rs has high-quality posts, but since it's inviet only, the cultural barrier to entry is high, and judgment itself is part of the community identity. It's more about signals like 'are you the same kind of person as us' than logical counterarguments. The community is small, so the same core users appear across many topics, and the reputation built there follows you around, making it closed off to minority opinions.
HN, on the other hand, isn't really a single community but a mix of many groups—startup founders, big-tech engineers, and independent developers like me. Depending on who happens to be reading at that time, the comment sentiment often shifts. And since there are so many readers, even if I say something on one thread, it's unlikely anyone will recognize me in the next.
So Lobste.rs feels like a hard place for an outsider to fit into.
It's in SQLite's documentation, and almost no one switching to Sqlite seems to be aware of it because no one discusses it in blog posts like these. I guess most projects switching to SQLite have very low traffic and no malicious users (yet)
Lobsters is invite only. That helps. :)
I'm absolutely no expert, I'm just reading about it now, but from the SQLite WAL documentation [0]:
> 7. There is the extra operation of checkpointing which, though automatic by default, is still something that application developers need to be mindful of.
and from their "checkpointing" documentation [1]:
> By default, SQLite does a checkpoint automatically when the WAL file reaches a threshold size of 1000 pages.
I'm only skimming but I see no mention of waiting for an idle time window.
Have you been successful in DoS'ing lobste.rs?
[0] https://sqlite.org/wal.html
[1] https://sqlite.org/wal.html#ckpt
>However, if a database has many concurrent overlapping readers and there is always at least one active reader, then no checkpoints will be able to complete and hence the WAL file will grow without bound.
>This scenario can be avoided by ensuring that there are "reader gaps": times when no processes are reading from the database and that checkpoints are attempted during those times.
Dunno, maybe Rails has a built-in workaround for this.
My workaround was to run a separate thread that monitored the WAL size on disk every second. If it went above the target size of 8 MB, my framework would enter "slow down" mode, where all reads and writes were artificially delayed by calling "sleep()", starting at 16 ms and gradually increasing the sleep time based on a few heuristics.
This allowed the application to have short gaps with no reads or writes, so the checkpointer could actually proceed.
This seems like a huge DoS security hole.
I encourage you to write a blog post about this. It seems especially relevant.
Seems to indicate a collection cron with a manual query to truncate can fix it. This is fun little things I like to learn. SRE Easter eggs.
Sometimes (maybe 5% or less) the request won't render at all, and you get a browser error page.
Today they ran into this bug, lost a bunch of voting data, and went into read-only mode for several hours:
https://github.com/rails/rails/pull/57128
I wonder how much of this is usual bugs which crop up during major database migrations, and how much is caused by choice of SQLite.
Maybe it's improved since I last used it, but to my knowledge SQLite essentially forces all writes to be serialised, at risk of data corruption otherwise.
There are tricks for improving the performance such as WALs, but that is merely a performance boost rather than genuine concurrency with things like row-level locks that you might find in other databases.
I guess if the whole thing is architected with a write-through cache that handles concurrent writes and deals with serialising all the writes, then it can be a single writer streaming changes through to the database, but then you still have a point of serialisation, it just will manifest itself slightly differently.
And SQLite is something that will give you constraints you will always have to consider.
Whereas running mariadb or postgres on the same machine would deliver similar benefits without a risky migration.
If your DB is small enough to run as a SQLite database, then it probably ought to have never been on a different machine in the first place.
There is a very happy medium between SQLite and a database on a different machine, one I am continually surprised to see people ignore.
Can you qualify "many"? SQLite easily handles 100k+ writes per second and it's not hard to have app layer code serialize and batch writes to take advantage of that speed. Concurrent writes require a ton of overhead and your system and code can be quite a lot faster and simpler if you just skip the idea of them altogether.
If you can do that, you can have infinity instances.
As an aside, on replication, eventual consistency is not a panacea:
If I make this comment, it's absolutely fine if you don't see it for a minute.
If I make this comment, it's absolutely broken if I then don't see it on my subsequent requests. I'll think the site has broken, and try to resubmit my comment.
You can relatively easily work around that one by pinning people to instances, but that's still yet another thing to consider, and those kinds of considerations add up when you're dealing with distributed systems.
However, this a single instance, it's way too early to talk about replication.
Given that just hitting their front-page is taking 6+ seconds, they've got a performance problem that needs fixing, and my hunch is that they have some kind of "last accessed" database entry, which turns every request into a write.
For example, one of the biggest optimisations that Hacker news does is that it serves logged out users from a cached copy of the front-page.
Logged out users don't care/notice about comment counts, they don't notice that it doesn't update as often, they can't be hiding articles so you can serve the same front-page to the millions of anonymous users and bots, and update that cached copy once and on a slower cadence than every request.
They could have one SQLite instance per user and then have a single sweeper that goes through all last writes and then replicates it to the main instance - eventual consistency fanned out across files
K1 acquired the commercial entity behind MariaDB Enterprise, but that's separate from the non-profit MariaDB Foundation. And there's literally zero risk of the MariaDB server suddenly going closed-source; as a fork of MySQL (which is GPL), this is not even legally possible!
[1] https://github.com/lobsters/lobsters/issues/539#issuecomment...
And in any case, https://github.com/MariaDB/ links to mariadb.org which is run by the MariaDB Foundation, not the commercial enterprise owned by K1.
Sometimes there are valid technical motivations to move away completely, but a nonsensical fear of a massively-used GPL database somehow becoming closed-source (with no surviving forks or continuity) is not one of them in my book!
- you have to use locks to prevent mangled write transactions, instead of the db handling it. Jepsen report is scathing
- its pretty hard to set it up in semi-sync replication mode, a.k.a. "only return from a transaction commit when the transaction is present on at least one other replica". Once you've set it up, you have to deal with insane things like transactions being visible on replicas before leaders, or even worse transactions visible on replicas briefly but never visible on leaders or other replicas
- every knob and feature contains subtle bugs, sometimes ones as scary as index corruption, that you only discover after it has caused you pain
I don't know why you would choose it unless you were already running it
The locking anomalies are a well-known and well-documented side effect of InnoDB REPEATABLE READ differing from the standard in certain pathological cases. The real-world relevance is minor, especially since companies using external write-through caches are already making much worse inherent trade-offs in that area. Regardless, MariaDB fixed it, while providing an option to revert to the old behavior for MySQL compatibility. And fwiw this only comes up because MySQL/Maria defaults to REPEATABLE READ; yet meanwhile the overwhelming majority of Postgres users are totally fine with the much weaker guarantees of READ COMMITTED, because that's their default isolation level.
Semi-sync has some odd behaviors indeed; there are conceptual trade-offs that must be made if you want synchronous-like replication without the full performance penalty! Do you have some novel academic solution for this that you can propose?
Your last bullet is lacking in details so I cannot comment.
Vitess. Also I often wonder if these system are on latest mySQL? I have read plenty of mySQL opinions that still based their experience during 5.x era.
If traffic grows they will find problems, then limitations, then move to postgres. And if not it's fine. Story old as time.
1. Stats are used by "Cost-Based Optimizers."
I think this is the most appropriate wiki:
https://en.wikipedia.org/wiki/Query_optimization
On my Oracle databases, I run the dbms_stats.gather_database_stats package procedure once a week. SQL Server also has stats for sure, and I would be surprised by a relational database that did not.
SQLite will quietly gather stats with each database connection. Running an explicit analyze devotes full attention to an otherwise piecemeal group effort.
2. I don't think rows are physically removed when deleted, instead they are marked. Removal will definitely happen at vacuum.
3. There is a backup command in the CLI that will make a transaction consistent copy of the database file, and there are lower-level C functions behind it that can be used with object code.
4. It is important to remember the many restrictions of WAL mode, including the impact on ATTACHing multiple database files: ACID consistency is lost.
https://sqlite.org/lang_attach.html
'Transactions involving multiple attached databases are atomic, assuming that the main database is not ":memory:" and the journal_mode is not WAL.'
I'm not interested in a rehash of Hacker News but I am interested in more tech news.
Anyhow on topic: I find it puzzling the whole "use sqlite in production!" thing.
- you're stuck on a single machine now.
- you don't have remote database access.
- you have to live with the inflexibility of sqlite's schema change capability.
- all writes go through a single writer.
- fewer data types
- fewer backup/security/high availability options
- foreign keys and strict disabled by default.
- you're single core in a multi core world.
It's not a terrible decision but when you can choose Postgres, why would you go with sqlite?
make sure you're not doing a bunch of migrations. that's the SQLite weak point that I know of & have experienced.
use SQLite - as if you're using a KV or better yet maybe a CQRS approach.
In any case I will take an invite if one has one because the community is solid.
https://lobste.rs/s/ko1ji1/lobste_rs_is_now_running_on_sqlit...
I hope they get it sorted soon, I'm rooting for sqlite.
---
I've heard "why not PostgreSQL?" a few times this week. It was even our original plan in #539! Well, it was a pragmatic choice in two different ways:
The person who volunteered to do the work used SQLite.
I don't want to use solutions that are bigger and more complex than our likely needs. Postgresql is my default for projects, but it does have the added complexity of being a separate service to run, tune, and maintain.
Citation needed. I love postgres as much as the next person, but it does have more overhead than Sqlite which is in an in-process db linked through compiled C code, it doesn't run as a separate server. Very significant difference that when you use Sqlite db, there is typically no db process other than your application logic, unless you implement the server yourself. If you don't want your application to have multiple processes (say, as a toy example) then it totally makes sense to prefer Sqlite just for this reason. Sqlite and postgres are different tools, they serve different purposes.
(Is this a typo?)
I do have to say that I've found lobste.rs generally a pretty slow site to render and navigate and I wonder whether that's the Rails backend, or underpowered hardware, or lack of caching. It's more feature-ful than a HN or Reddit though with much lower usage so I wonder what it is.
https://sqlite.org/rsync.html
... and is installed and available:
"Ordinary rsync does not understand SQLite transactions. Rsync will make a copy of ORIGIN into REPLICA, however the copy might not be consistent. Parts of the copy might be from one transaction, while other parts might be from a different transaction. The database copy might be corrupt."DHH has made a big deal about using SQLite in rails
I'd like one too if someone wishes to invite me as well, Thanks and have a nice day!
You are awesome mate!, have a good day :-D
Lobsters still got a pulse, largely.
As in: I just don’t understand how IRC works…
Then again, maybe they like the ability to critique your posts without you being able to rebut their opinions. It's an effective tactic in online echo chambers.
This feels antithetical/dystopian/elitist compared to what online communities are, for me. I think it's ok for some communities to be like this, but I don't think it's the future of online communities.
I think that was one of the big mistakes/oversight that came with the Internet..
I am not so thankful. But it is coming from my view of the current world.
I think open Internet for business is good. But open Internet for communication was bad. I cannot think of one good thing that have resulted from an open internet for communication/discussions. It has turned out to result in a collective, perpetual state of seizure for all humanity.
I am someone who had to learn stuff from random encounters with books in the 90s. Even then, I would not have it any other way...
Deciding who is in or out is the fundamental basis of any community, that's freedom of association. That's what defines a community. A place anyone can enter is a train station bathroom and that's what the logical terminus of the open internet seems to be.
Although the statement "invite-only only is the future of online communities" isn't super well defined (what does it mean to "be the future"?), so I think the comments are debating past each other.
It's also not open access. You have to run an invisible tightrope so its algorithms won't automatically shadowban you and then you have to run a barely visible tightrope to get enough karma to not have your posts automatically removed from every subreddit.
AI posts are usually the most commented on. Linus Torvalds’ comments saying that LLMs are actually useful is still on the front page, tagged as vibecoding, and has a lot of comments from people mostly disagreeing with him.
Lobste.rs is more of a monoculture than Hacker News. If you get downvoted enough they stick a banner on the top of the page inviting you to delete your account. It has a nasty side effect of driving away good contributors who don’t align perfectly with the hive mind. It’s a site where you learn to keep your mouth shut if your input doesn’t agree with what the core users want to hear.
0: deleting comments of people who aren't interesting from my view and highlighting ones who often say things that are interesting, the user I'm replying to is one of the latter
There are some interesting comments there from some people who don’t spend time on other sites, but your assessment is similar to mine
My comment about being anti-AI above may not have been clear. They actually talk about AI a lot and AI threads attract a high amount of activity. The difference is that insightful or honest comments don’t have much of a chance because there’s a clear popular opinion on the site and any information that doesn’t fit isn’t welcome.
The giant soupy mess of opinions on HN can be tiresome, but it’s nice to get a wider range. HN is its own little bubble but at least there’s some diversity of experiences to be found.
That said, I don't believe it's as much of a hivemind as you've portrayed. I've seen dissenting opinions voiced there which weren't downvoted (or at least not noticeably), while the same opinion in a similar thread here on HN would be downvoted and flagged. So I don't think that getting downvoted enough so that you're invited to delete your account is a common occurrence with productive members.
It is! It’s a rite of passage on the site if you dare comment anything that someone might disagree with.
There have been several high profile cases of valued members getting fed up with it and leaving the site. One example where even the community was starting to question if this is really what they want: https://lobste.rs/s/zp4ofg
If you got into my group of death metal people, and suddenly started talking about Oasis... we would invite you to get out haha.
Same with patriots.win . They may be a group of right wing biggots, but hey, they did their platform and are happy talking there.
Some serious cognitive dissonance here.
I think a lot of people like this idea of a close-knit clique of like minded people, but the communities tend to pick up toxic qualities where membership becomes a game in itself. Anyone who doesn’t fall in line with the established in-crowd gets pushed out via the Byzantine rules that the in-crowd uses to maintain their position.
I still visit and participate, but I’m realistic about what it is. There is some value there, but you have to navigate it like a small subreddit where the moderator keeps tight control over what can be discussed and indirectly or directly influences the allowable opinions on the site. I really liked their original ideas about having an open moderation log and clear rules, but I’ve seen more than enough questionable moderation decisions and selective application of the rules to accept that it’s not as pure as I thought it was going to be.
edit: my response isn't about lobste.rs. I don't have an account there.
I joined their IRC channel last week and it's been rather chill.
I do like the invite-only nature of the site though, keeps quality high...while making some folks tyrannical.
My intuition says it's to do with Status Signaling theory .
I love it.
That's not necessarily a bad thing, but honestly, as a foreigner, I have limited English vocabulary, so I end up getting stressed out over being judged as AI. At least here, the admin DEN kindly understood my situation and let me participate after getting permission. But when I look at subreddits and other communities, the stronger the monocultural management, the more fan-like sentiment tends to grow, making it harder for minority voices to speak up.
I'm in favor of LLMs (since being for or against LLMs inevitably depends on regional context—I personally find LLM translation incredibly convenient and helpful), but there are moments when I find it hard to argue against blanket opposition or certain topics. I thought about why that is, and I think it's because Lobste.rs has high-quality posts, but since it's inviet only, the cultural barrier to entry is high, and judgment itself is part of the community identity. It's more about signals like 'are you the same kind of person as us' than logical counterarguments. The community is small, so the same core users appear across many topics, and the reputation built there follows you around, making it closed off to minority opinions.
HN, on the other hand, isn't really a single community but a mix of many groups—startup founders, big-tech engineers, and independent developers like me. Depending on who happens to be reading at that time, the comment sentiment often shifts. And since there are so many readers, even if I say something on one thread, it's unlikely anyone will recognize me in the next.
So Lobste.rs feels like a hard place for an outsider to fit into.