> Consider though that in this community 99.9% of the time others have solved your problem before. You could reinvent the wheel (which is arrogant), or you could do some research to see which architecture has been successful for others.
The best way to learn something is to come up with it yourself. If you base your architecture on what you read in books or elsewhere, you probably only have a limited understanding of it. And that's especially bad when you're talking about the architecture of a program.
So maybe a better idea in terms of the end result is to first come up with your own solution, implement at least a functioning prototype, and then do some research, to improve upon your idea based on what you have discovered (or throw it away, in the worst case, but of course keep what you've learned doing it).
The best way to learn something is to come up with it yourself.
That's been bugging me, and it's been bugging me because it's not entirely right. It's not entirely wrong either, though.
Given an undirected graph, in which you need to calculate the shortest path between two vertices, would you slog it out or would you just find a shortest path algorithm?
Now move that up a level, from feature implementation to feature design -
If you had to federate identities between two directories, would you hack something together, or would you have a look to see if someone else has done that before? Assuming the latter, you'd probably discover OpenId, OAuth, WS-Federation, and SAML. Would you then use one of these or try to roll something that models their behaviour?
Take that up all the way until you reach the age-old build vs. buy debate, and the connected enterprise and so on. Building something just to understand it is not making much sense to me.
> Given an undirected graph, in which you need to calculate the shortest path between two vertices, would you slog it out or would you just find a shortest path algorithm?
Probably just find an algorithm. This is because my goal was not to learn about shortest path algorithms, rather it was to find the shortest path. Whichever algorithm I chose and however I implemented it is of minimal concern - if it turns out to be bad, I can probably redo it without significantly affecting any other part of the program. I claim that, if, on the other hand, you wanted to learn about shortest path algorithms, it would be better to first at least try to think of a solution or two, then go for the books.
Speaking of software architecture, given the profound impact a solution will have and how disastrous bad choices can get, it should be your goal to learn about whatever you're dealing with.
> If you had to federate identities between two directories, would you hack something together
Right, that "try it yourself" part can get very big and impractical. But the idea still stands: it would be better if you first tried to think how a solution would look like and what it would do, in very general terms. Are the two systems very similar and a solution would be little more than mapping one thing onto another? Or are the systems very different and complex translations would be have to be performed?
Once you do that, you can have a look at any existing solutions, and almost certainly you'll have a better understanding compared to if you didn't do the thinking part first.
It is also important not to switch the order of "thinking" and "looking". If you first look for existing solutions, then your thinking will be biased by what you have seen.
You're describing semantics of moving from a solution concept to an implementation. How you get there is entirely up to you.
What I was suggesting is that a design that separates concerns will obviate the need to keep the entire thing in your head. This is problem solving through the composition of discrete components. AKA managing complexity.
You seem to be in love with the idea that a problem can be understood by decomposing it into sub-problems, and considering them individually.
Where I could agree with you is that a solution to a problem should be understandable in pieces, and then composed out of those pieces. Where I don't agree is that I think piecemeal understanding of the problem is likely to yield suboptimal partitioning of the problem into abstraction pieces. Following your train of thought you will end up with a solution that is locally optimized, but globally inefficient, because the problem was not partitioned correctly. Proper partitioning, I think, requires global understanding of the problem, with enough detail of what matters. The problem with global understanding is that most worthwhile problems do not fit in the head easily, but the solution is that coding pieces of the solution allows a more efficient way of reasoning about the problem, eventually leading to a global, or at least global enough understanding.
In other words, I worry you succumbed to "just design it correctly, and it will work well" fallacy. I call it a fallacy on my own experience, but also on the "Mythical Man-Month" book. I'll give the right quote in a moment.
I might be wrong in how I see your point, but if I am right you will hit some serious problems down the road. I don't think I can convince you now, but it would certainly be worth the effort for you to remember this conversation and think back to it if you do in fact hit the problems.
[EDIT] And here's the promised quote from "The Mythical Man-Month", by Frederick P. Brooks, Jr., 1975, to illustrate the "just design it correctly, and it will work well" fallacy:
---------------
I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash he broadened my whole view of the software process. The metaphor shift was powerful, and accurate. Today we understand how like other building processes the construction of software is, and we freely use other elements of the metaphor, such as specifications, assembly of components, and scaffolding.
The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be accurately specified in advance, and too complex to be built faultlessly, then we must take a radically different approach
Let us turn to nature and study complexity in living things,instead of just the dead works of man. Here we find constructswhose complexities thrill us with awe. The brain alone is intri-cate beyond mapping, powerful beyond imitation, rich in diver-sity, self-protecting, and self-renewing. The secret is that it isgrown, not built.So it must be with our software systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development.
That is, the system should first be made to run, even though it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit it is fleshed out, with the subprograms in turn being developed into actions or calls to empty stubs in the level below.
I mentioned this below - I've been programming since 1986. It took me the last 7 years to really nail down that this duality of perspective (high-level, complete system vs. low-level, one aspect) was common across all successful projects I've lead or worked on.
By success I mean something that not only meets the stated requirements, but is also maintainable, extensible, secure, scales, and is available.
The best way to learn something is to come up with it yourself. If you base your architecture on what you read in books or elsewhere, you probably only have a limited understanding of it. And that's especially bad when you're talking about the architecture of a program.
So maybe a better idea in terms of the end result is to first come up with your own solution, implement at least a functioning prototype, and then do some research, to improve upon your idea based on what you have discovered (or throw it away, in the worst case, but of course keep what you've learned doing it).