I'm not sure about your point on schemas. When using relational database you must have your schema in two places: 1. database and 2. ORM - the second one is enough to avoid most of problems.
What's the point of having schema in database then?
For one, your DB schema can define constraints or procedures that are triggered inline to the work being done. If building those into the application layer -- not including a middleware -- you are going to pay in network IO and probably wire protocols to guarantee constraints or execute procedures. A middleware could reduce some of the costs, but then you are again moving some logic outside of your local model.
1. Your ORM can derive it's schema from the database's.
2. I think the main point of the post was that you can run ad-hoc SQL queries no matter how much you've denormalized. You can't necessarily do that with a NoSQL database.
Yet you are still using schemas in 2 places : not DRY , SQL just doesnt fit OOP design. Most of languages today have strong functional capabilities , that makes SQL obsolete , they have functions , event systems , RDBMS exist because people use to query those systems directly , does your users log into your database directly ? no they connect your database through middleware, that's where the job should be done.
DRY is "Don't Repeat Yourself", not "Don't Repeat Ever". If your ORM fully correctly derives everything it needs from your database schema, that doesn't count as a repeat. If your ORM needs a couple of hints, but those hints are indeed extra information that your DB can't have ("this isn't just a string, it's an IP address") that doesn't count as a repeat. If you personally typed your schema into a database creation script and then you personally also typed the same schema into your ORM specification, only then are you violating DRY.
Code generation is a powerful DRY tool, not something it suggests avoiding!
I say that independent from the question of whether you should be using an ORM at all. If you are going to use it, either your ORM should come out of the DB or be responsible for creating the DB, either is fine, as long as you're not saying the same thing twice.
How do I combine data from multiple log files? How do I tweak my "queries" without scanning all of the data again? You get this kinds of things (joins, indexes, etc) for free from RDBMS. If I'm analyzing log files I have to write my own code to do it.
I'm not saying RDBMS is the best solution for everything and neither is OP. But it's appropriate when you don't necessarily know every way you want to access your data up front.
Even when you use OOP languages some tasks are really badly suited for OOP, so then just code it like it was imperative or functional. The relevant example here is reporting, where SQL is one of the most suited languages for this task. Use the right tool for the right job.
A fixed schema in the database means you only have one schema to deal with. But in schema-less DBMS you really have as many schemas as changes you've made. If you add a field to new records coming in, you now have 2 schemas. Every change you make is another schema.
Your ORM likely represents the last iteration of that schema and then you have code to handle all the past iterations.
So the point of having a schema in the database is avoiding that kind of hell.
> in schema-less DBMS you really have as many schemas as
> changes you've made.
Unless you migrate data with each change, which is actually easier when you don't have a static schema. The choice to do "lazy migrations" is the unusual, not typical case, as it has all of the implications that you've mentioned. Bottom line: Data normalization doesn't suddenly go out the window just because it's MongoDB.
Usually if you have a static schema you also have SQL, which makes migrations a lot easier. If you're migrating everything with every change you might as well be using a static schema, there is hardly any downside, unless you have more rows than God.
My only personal opinion is that data is much more important than code. I want to guarantee that my data is in the right structure, with the right types, and right relationships. If I have that, everything else is easy.
> Usually if you have a static schema you also have SQL,
> which makes migrations a lot easier.
I'm not sure I follow. MongoDB doesn't have a static schema so there's less work to do when migrating data. No temp tables, no DDL, no disabling of triggers, etc. So my definition of "easy" in this case was "fewer things to do", not "requires less expertise."
> I want to guarantee that my data is in the right
> structure, with the right types, and right relationships.
We know that having the right types, structure, and relationships are valuable, but since there is no context here, we can't know whether they're more or less valuable than other factors. (Note we've already assumed that the data is easily modeled in a relational structure AND in a document structure... If we're wrong about that this whole conversation may be moot.)
> MongoDB doesn't have a static schema so there's less work to do when migrating data.
That depends on the operation. If I want add a column and provide a suitable default, that's a one-step process in a SQL database. Even better, I can do it from a GUI tool and generate SQL script ready to use in production. So my definition of easy is both fewer things to do and less expertise. Even taking a more complicated yet common example of taking a single field and turning it into a collection or table of values -- I think the SQL approach is still going to be less work. But I think the more complicated of a transformation you need, the less of a difference there is -- once you start writing a lot of code it doesn't matter if you are modifying a static schema or a dynamic one.
> But I think the more complicated of a transformation you need, the less of a difference there is -- once you start writing a lot of code it doesn't matter if you are modifying a static schema or a dynamic one.
I agree with your general points but disagree with this. Since SQL provides excellent tools for set operations and integrity constraints even very complex migrations are simpler if you have SQL + a schema. SQL features like temporary tables, window functions, inserts using a select as source, CTEs, all help out when doing the really tricky transformations.
Th exception is of course if you have millions of rows of data which you need to do a complex migration for. Then I do not think it ever can be easy, schema or not.
What's the point of having schema in database then?