What is the industry/production grade solutions or if you have already any experience please share it. Thanks

3 points

In AWS, you can use Aurora to scale reads access over multiple instances. It’s easy to setup and you can use scaling policies to follow your load. You need to check your billing, it’s not the cheapest service.

permalink
report
reply
3 points

IIRC, MySQL (and PosgreSQL) is pretty much limitted to a write/read-replica sort of horizontal scaling. Other SQL engines have better support for multi-master configurations.

However, these types of configurations are usually tied to licensing - especially for Microsoft SQL server and OracleDB.

As another commenter suggested, there is Yugabyte and CockroachDB as well - of those two I think CockroachDB is the more mature product. And they’re one of the fiercest competitors for the company I work for too.

I cannot speak to “Battle Tested-ness” of CockroachDB, but given it’s been around for a few years now, I don’t think it’s quite as risky as other comments have indicated. Also, they’re doing something right if we haven’t been able to convert many CockroachDB customers.

permalink
report
reply
3 points

Scaling horizontally is never trivial, can you share more about the requirements?

Usually application level partitioning, but it’s hardly repeatable. Cloud providers offer solutions like Spanner, that cost quite a bit. You have new SQL DBs like yugabyte or cockroach, which are not exactly MySQL, and are not really battle tested. And noSQL like MongoDB or Cassandra, which are, well, not even SQL, but are proven technologies and scale quite effortlessly.

permalink
report
reply
2 points

I work for another distributed database company. I can say that it’s much harder to convert cockroachDB customers than Yugabyte customers. Given that, I’d think that CockroachDB is likely the more vetted solution. Sure it’s new (2017), but it’s not THAT new.

permalink
report
parent
reply
2 points

If you truly need read/write to scale, multi-masrer clustering with MariaDB Galera is probably the best way to do it. They (MariaDB) also sell a load balancer/query proxy Max scale that can do a lot of surprisingly complex stuff (like publish new data to Kafka or centrally ship binlogs from one place to multiple read replicas).

However generally my advice is that if you’re finding yourself trying to build a big relational database and writing to it a significant amount of times, it might be time to consider a different or at least modified architecture. Especially if your use case starts to scale to more than what Galera can handle. At some point, all these solutions become eventually consistent the more you scale. And if you are willing to accept eventual consistency, there are some clever ways to do storage using things like queueing, batching, caching etc that can scale horizontally much further than any relational database can.

permalink
report
reply
1 point

Wow, thanks 👍👍👍 this will help me a lot.

permalink
report
parent
reply
2 points

I hope so! Looking forward to the writeup at the end!

permalink
report
parent
reply
2 points

For on-prem we use a Galara cluster. It’s managed by ClusterControl because we needed something with enterprise support.

permalink
report
reply

DevOps

!devops@programming.dev

Create post

DevOps integrates and automates the work of software development (Dev) and IT operations (Ops) as a means for improving and shortening the systems development life cycle.

Rules:

  • Posts must be relevant to DevOps
  • No NSFW content
  • No hate speech, bigotry, etc
  • Try to keep discussions on topic
  • No spam of tools/companies/advertisements
  • It’s OK to post your own stuff part of the time, but the primary use of the community should not be promotional content.

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

Community stats

  • 2

    Monthly active users

  • 74

    Posts

  • 253

    Comments