2014. március 8., szombat

Prelude to mission statement


@Ethar Philosophical? Nonono, sir! If you want philosophy on my side, check this :-)
I could also explain in a few paragraphs why and how your ultimate concept, the monetary evaluation gives fundamentally wrong motivations, and as side effect, makes us destroy our own habitat if we don't change it, I would say THIS year. That would be philosophy, but also useless, because it is too far out of your comfort zone to follow. So forget that.
(No offense, this is my experience in long years, the simple explanation is in the lecture - but if you also realized that the Hitchhikers' Guide is a fundamental book on philosophy, accept my apologies.)

I seem to be the one here who talks about actual technologies and lessons in IT stories, not merely repeating the "money makes the world go round" mantra (because that applies to your world, while in mine it is the laws of physics, by the way). I have spent most of the past 20 years, I go to work every day now to design and build a well-architected framework myself, not to sell or evaluate it. I don't want to be offensive with this, but I would like to draw the clear line between our roles, because that defines our experience, and therefore the concepts that we accept without arguments.

When I talk about ROI, I don't mean a system that SELLS well, but the one that SERVES well. When I talk about modularity, I mean how much we must refactor the original requirements, how much we must forget the features and focus on the architecture to make a system that survives, or better: prevents some of the change requests after release, or even in the test period ("I did not mean that!"). _I_don't_care_ that with signed specs in the contract we can make the customer pay for that garbage, and pay again for the new version because that gives our PROFIT, and that is we work for... that is what you suggest I should work for!

When I talk about platform independence, I don't mean multi-platform toolkits by giant companies or a global bunch of script kiddies, but portable concepts that I alone could implement in months, and which payed back the effort in that project alone (but could be used anywhere). When I talk about separating the concepts from sources, I don't mean UML dinosaurs and multi-language code generation but a runtime that allows me setting up the type system and the application itself in configuration files (partly typed in C#, Objective C and Java so far, the planned final version to be done in plain C). Sorry, nobody spent enough time on thinking, so I had to write it :-) Can I sell this to any consultant, get a sponsor? No way. There is no business in doing a job really well in IT (ask Microsoft about Windows XP).

Personally, I understand your change, and I understand the "product must be sold" requirement - but still I chose the harder way, because I realized that no ROI-focused projects can answer the questions I ask. This is called basic research which I do behind all actual tasks that I do. Today I can answer questions that others can't even see. Unfortunately, that makes them unable to understand it either - but now I gradually can show working elements, and that changes the situation. Sorry, I can't do the same here even if anyone would spend the time, because this is closed source. I was not good enough for universities, when I asked... and it is better in this way: the ROI and time pressure to play my games behind an actual and tough task is my dearest enemy :-D



@Edward - First of all: you always compare the "unquestionable" monetary ROI with "opinions", with slight sarcasm on the second. You seem to forget that the mere definition of real innovation is to break some rules and habits, so they are fundamentally against "common experience", and there you have nothing else but opinions. To me, your ROI concept is a fundamentally wrong opinion, but because it is globally accepted, it affects the whole world and myself. I must adapt to it to survive, but I don't let it go above this layer.

Opinions to you seem to be equally gray in the shadow of "responsible thinking" and monetary ROI, following them merely a trial-and-error story. In most cases, you are right - but true innovators are those who can find new paths, evaluate opinions, make sensible mistakes, "waste money" to get experience in the good directions. And finally, perhaps after reusing long list of previous "failures", bring up something that is marketable, and has financial ROI. This used to be called "science" - but today you don't find a university that is not fighting for money, and lost the best brains who (in their personal quest for money and holiday on Hawaii) first do their startups, and after some failures go work for "innovative companies" like Google.

In your career with giant, stable companies, it is not likely that you have ever met with true innovators, and among startups, you should check a thousand to see one. You are statistically absolutely right, this makes the world go ROUND. But please accept that I talk about the exceptions that move the world AHEAD!

2014. március 5., szerda

Innovation and ROI

LinkedIn

@Edward - yes, this is how it goes: we all follow our opinions, and share them for reviews from others who have different ideas. And this is cool :-)

My personal opinion is that selling a product for profit and solving a problem (translate: financial(!) ROI and true innovation) requires opposite mindset, Google's above mentioned mind farming strategy clearly proves that to me. I also think (based on what I have seen around me in the past 20 years) that 99% of the source codes today are simple N-th repetition of simple tasks (GUI event management, serialization, logging, event dispatching, etc); housekeeping; monkey-typing (or in "better case", generated codes), M-th scripting language and the yet the newest, smartest GUI frameworks. Almost zero new idea, innovation, because that requires tons of FAILED experiments, broken deadlines and abandoned development paths beforehand. There is no other way, only in today's sci-fi trash.
Not to mention true knowledge and solution sharing, which is a totally stupid idea from profitability and patent wars point of view.

This is my opinion. It does not make me really happy - but gives me very strong motivation to cut this Gordian knot, and right now I have a quite cool battle axe for that ;-)



Innovation? A new kind of TV??? Well, I think we have different view on this term. For intro, you may check this...

Apple's great innovation was to ... er... reuse the hardware ideas from Xerox PARC (GUI, mouse, tablet, etc.), and put the truly great Objective-C language into the heart of its environment (which is by the way, based on Smalltalk... another "reused idea" from PARC).

Apple followed a very strong vision to make a feature-rich but user affordable hardware (at the beginning it looked impossible), targeted at education first, set a very high standard regarding the GUI to minimize the user learning curve: work as expected. Again: Apple's HIG is a piece of solid gold for any app designer. And this worth the money, builds a faithful user base, so gives the ROI. For a while.
At the same time PARC went down the drain because they had no idea how to make money on their innovation, have not built patent stack, let others "use" their ideas, etc. They just did a great job on a single task: invent the technology required for a paper-less world. Unfortunately WE are unable to imagine that world...

I can tell you my story, how I was able to type in a GUI framework with configurative data binding, and a statistics viewset on an OLAP cube in Objective-C, etc. and so build half of a complete (and currently live) project within 5 months from the first line I wrote in Objective-C (including the complete month that I spent on reading through their docs). Because iOS is truly a great environment, and indeed a well-architected framework, thanks to the great coherence among the language features, data management, GUI features, etc.

But instead of Apple, that is because of the true innovation done at Xerox PARC in 1970!



@Gary - YouTube is a good friend: you can find lots of interviews and documentaries about that era and with Steve Jobs, I have spent quite a few hours watching them.

He was a very conscious businessman, with a strong faith and charisma. You can watch him setting two targets for his group: a price tag and a deadline; the team is to find out what they can squeeze into this box. Is this innovation? Come on...
I think there are several people with these abilities. But that part of your story is true: he went to PARC, and what he saw there just flushed away everything that he wanted to do before, and became obsessed by those ideas. He was sensitive to the potentials, could motivate people with his faith, and present the result in a unique way. True talents of a great salesman.

What I dislike in your story is the "rock on the ground": you seem to be blind to the real work. That rock came from a mine after several years hard work of an elite group. I repeat: they invented an object oriented, message based programming environment (while "an OS should be written in C"), built the first GUI system on it (when others played with a console); created a computer network, laser printer, even the concept of tablets. In 1970, do you remember? The capacity of those machines were well under the CPU of your dishwasher! The results of PARC inventions costed fortunes, and were surely useless, _dead_slow_. When Xerox watched the possible ROI, that was simply zero, they felt that all the money they spent on PARC just went down the sink.
It required someone who started to believe in it, and this was Jobs. He could NOT spend the money and years Xerox did to get these "rocks" himself; and if we add that cost, he could not start his venture. But he had the chance to "take" what he saw, and start building a company to make money on others' results, like a multistage rocket in IT business. He also took a giant risk, but at least he had the chance to win in this way, and so it happened. And we all should be grateful for this, because we actually live in his dream.

But I am a programmer and not a salesman, so I don't care about the famous CEOs (they are the guys who set goals and give resources), but the guys who _do_the_job_: who invented the fundamental elements of a new world, though they looked ugly and useless. And in this case, these are the guys at PARC, so my real respect goes to them - without neglecting the importance of Jobs-like figures. And the same way, about framework architecture, I ask the opinion of an inventor and not a salesman.

@Edward - the question is very good. I will try to summarize my answer to it in a way that you can probably accept (even if you don't agree); a kind of mission statement for my whole career. I need some time for that, though :-)

2014. március 2., vasárnap

Mind farming

LinkedIn

Investment to me means time and resources, from all parties: architects, developers, administrators, end users, and those whom the end users provide the service to.
Return means not only the successful service, but the core innovation in the system, and the more complex understanding of the tasks personally.
Translate them to money to me means throwing away the real (and yes, quite complex) lesson for cash - even though I know that I need salary today, and that comes from selling my work. This is why I say: I don't want to make decision(!), and accept financial reasons(!) - but I don't let them poison my analysis either.
As analogy, I am a true geek, so I prefer reading Asimov instead of watching any new sci-fi "blockbuster" trash. By the way, my vision is quite precisely written in the Foundation stories.

You talk about the financial sustainability of a company with successful _products_. I talk about _technology_and_innovation_, and I see that our world badly misses that, because all our smart people dig for profit instead of looking forward.

Today we praise "startups", and seek for innovation from them. Why? Big companies do have smart people, give them lot of money, but... yepp, they work for monetary profit, financial ROI. Startups have vision, not ROI plan (listen to Guy Kawasaki), they are allowed to fail 20 times per one success. Sorry, at the edge of technology, this is how it goes, and new things only come from here. This is my place, this is how I classify a system well-architected.

The success of "innovative companies" like Google comes from that they host both approach under the same hood. They have strict business plan and work for monetary ROI in their _products_, but let their employees run their own "startups" in fixed percentage of their times forgetting even the smell of ROI, to work on technology and innovation. The company harvests the 1 percent innovation that pays back, and another 9 goes to patent stock, I presume. This seems to be the winner so far, not to mention the fame for any creative people that they are praised and payed very well for their pet projects. Smart mind farming... :-/

I do this model as a one-man show. I have worked on various fields, at several companies and in many languages, environments in the past 20 years. You can explain the lifecycle of a product to me, but the same is true for my story: I applied and refined my concepts behind all jobs that I have done (data mining, spam monitoring, national data management system, log analyzer, home automation, public touch screen terminals, ...). I can tell the motivations behind the languages, toolkits, etc. by just playing with them for some days. I have thrown away tons of code that better salesmen would make fortune on, because they were not "well-architected" enough, and he errors started poisoning dependent codes. I have lots of failures in small companies and short projects, and some success, all in longer term projects, where I had space to run my way.

This really worth 20 years, gave me joy - and yes, pays back to my companies more and more, because today I am really fast. For example, I could type in a configuration-based web GUI framework literally in weeks (because I have the underlying management framework that took another some months now). It was 1.5 years to invent the last time at another company, but that also has run fine in the past 10 years, so I think that also has fine ROI. :-)

... and no, I don't sell it to Google, because I bet they would pay me NOT to do it. I have more important vision: leave a habitable planet behind us for the human civilization in the next centuries. This is what we blow up while concentrating on ROI, despite of the best effort of our famous world-savers (rich, smart but shallow-minded businessmen or dreamers). Sorry, I went too far again.

2014. március 1., szombat

Primary objectives

LinkedIn
Software Architects and Enterprise Architects Group
How would you describe a well-architected system?



@Edward - thank you for the details. I was quite sure that behind the emphasis on ROI, you have a serious technical background which is essential to properly estimate it. I am also sure that your insights and objective analysis do help in making the long postponed hard decisions, and you can sell them to the management.

I never got close to big business like you. I am an obsessed programmer and analyst, and I have weird ideas about programming, life, money, etc. - that always stopped me in selling myself. My favorite task is (and they find me all the time) understanding new areas, refactor existing solutions, and/or build complex systems from unclear requirements. I know how huge difference can be between two systems doing the same thing: one that answers all current questions and another with open structure that allows asking and efficiently answering new ones.

I spent all my time searching for the absolute minimum code to solve the same task, at various areas (from data mining, telco, to government data management systems, etc.), in different languages. I also spend 1.5 years "at home", refining my concepts about programming. Today I am able to forget about the actual task and focus on the structure instead; I can feel when I must stop before writing codes that I will throw away in months.

I always have to face with the situation that somebody important does not like what I do - but I am open to discussions and do my best to explain my reasons. I am not a manager, I don't make business decisions and I work for the money they give me. My job is to design and present the best architecture I can, and fight for it as long as I can - and then accept their decision that also includes the business limitations.

This is why I work today for a big company as internal developer: they have more patience for a Don Quixote like me. My motto is: "I know that this is not what you asked. This is what you need, but thought it is not possible." :-)
I do my best to prove it.

@Lorand - you can do a lot things but first you need to have sponsors that belief you can help them, and profitability was the thing I used. As a result I had my sponsors and I got to do a lot of fun system initiating things; most don't get that opportunity. 

I also was the CEO of 2 unfunded startups, each lasting about a year. We merged in a AI productivity focused tool that could generate the same rules in 4 different programming languages (a friend developed it) and were able to create foundation insurance (1st) and security (2nd) products in an amazingly short time. We had a lot of participants but no money to sell the products, When the effort of keeping the company going (in the government eyes) got be too much, we quit the self funding route. But I did learn a lot about AI/Expert Systems or whatever you call them.

@Edward - or accept that no one shares your faith in a single project that lasts for 20 years (literally), because that much time is required to do all the mistakes, learn all the things that are essential to make it work. So you become your own sponsor: work for lower amount of money (or just nothing, if you can afford it for a while), in an environment where you can make time for yourself to go ahead. And shamelessly fail on several projects, only for the experience. Not a ROI-friendly approach, nor a dream for wealth improvement, but what else can you do in a system where personal experience gain is simply not represented (not translated to money)?
[ My only objection to your ROI-first approach here is that we have to go this depth to reveal that you do have exceptional technical/management experience for building your ROI analysis! (sorry, I have not checked the whole conversation) This is true to you, but according to my experience, almost none of your readers, including me. ]

Then, at certain point of time, you step out and say: "you gave me a task, and I am in delay with it - but I see your whole environment, and I spent 90% of my time on solving the root cause that initiated the original request." At this point the key factor is: what can you put on the table? In my case, this is my Holy Grail, consciously tailored to the current environment's needs and resources. I hope it will be enough :-)

To the case you mentioned: source code is merely a serialized form of your concepts; if you can design the concept properly, the language is just another text template (I had a short experiment with a framework that generated half of its codebase). You must be absolutely sure what kind of code you should generate, and exactly why that part?

My favorite example are the click-and-drag GUI builders that generate code for different GUI frameworks, and they are proud of it!!! OMG...
They don't even realize that they build an object structure, then serialize it to source code, then run that source to build up the same data structure again runtime. Yeah, you can edit the code - but then how to separate the generated part from your logic? Complex questions, hard work again - on a fundamentally wrong question. They seem to forget why XUL / XAML / GWT UIBinder / .rc files in the golden ages / etc were created, and what that means. Epic failure.

2014. február 28., péntek

The hidden costs

LinkedIn
Software Architects and Enterprise Architects Group
How would you describe a well-architected system?


@Gary this is exactly my point of view (based on similar experience).

Taking one fabricated number as the key measurement (without questioning how effective it can be in most cases, in our economy today!) is dangerous. It hides the precision of the method creating it (not to forget about that not-really-Churchill-quote saying "I only believe in statistics that I doctored myself"), and the likely motivated reasoning behind. And it does not replace a conscious risk analysis for the future.

@Edward, I have checked your site, and we could argue for years without convincing each other, because we have totally different experience. I assume you have helped lots of companies with your strict rules as a consultant - I have seen the same rules breaking the neck of systems from inside, as an architect. I accept the importance of your concept with one exception: your statement of "no exception". I think the ROI analysis-based barriers can and should be broken _sometimes_, after conscious and responsible negotiation. The same applies to my side: the best architectural ideas can be put aside _sometimes_ because of the too risky ROI estimation - after conscious and responsible negotiation.

My mantra, the "conscious and responsible" is missing lots of times both on management (Holy ROI) and designer/developer side (Holy Architecture, Holy Tools, Holy Patterns, etc.) - and without an open communication, it becomes a power or marketing (polite way to say bluffing) battle, a poker game.

Yes, even conscious and responsible people can make mistakes - but at least they can identify the causes and make better decisions in the future, adding ROE (return on error :-) ) to the equation. To move forward, we _must_ fail several times before reaching a higher level goal, so in several cases we _must_ pay the price for nothing (but experience). "Experience is what you get when you don't get what you wanted"... I got this from Randy Pausch, a man worth knowing about.

If we all focused on measured and stable ROI, we would have nothing from our current achievements in science, technology or society. IT should be something like "controlled and monitored creativity"... No surprise that I work today on SW Quality Assurance, with concepts like: "the best code is _not_written_", and work literally years on a framework that actually minimizes the required amount of code :-)

@Gary, back to the example: this approach does not ruin America only, but threatens the entire world today. Space exploration is inherently dangerous, and we stall because we are blinded by Star trek-like movies (main heroes always survive) and afraid to lose precious human lives, though it is part of the real game. 
On the other hand, we play with technologies that can really destroy countries, have global consequences, with the same shallow thinking, and _that_is_really_bad_. I think this lecture is worth listening to... http://fairewinds.org/podcast/fukushima-daiichi-nuclear-accident-ongoing-lessons

2014. február 26., szerda

How would you describe a well-architected system?

LinkedIn
Software Architects and Enterprise Architects Group
How would you describe a well-architected system?


In my experience, I have found a useful rule that I don't remember reading anywhere, so I consider it "my own": the 90-9-1 rule.

I always try to design and implement systems that spend
  • 90% of their lifetime in a state that requires the 1% of effort. To use the system the user does not have to "learn" too much (recommended reading is the Human Interface Guide from Apple). The system maintenance (like log rotation, memory management, automated notification on anomalies etc.) is designed properly, so the administrators don't have to "ping" the system: "if I know nothing about it, then it runs fine";
  • 9% of their lifetime in "administrative state" that requires 9% effort. This is the administrative part of the application that the user has to learn to use (but this part is validated and even the admin should not crash the system - or at least not easily :-) ). These are the planned upgrades, handling change requests, etc. This is normal for any live system, and should be considered during design and development, supported by the architecture and the running framework;
  • 1% of their lifetime in "exceptional case" that requires 90% effort. This is a major structural change request from the customer, where both parties has to put serious effort in design and implementation. The system should support finding the boundaries: how far the side effects may reach. This is also a major problem in the framework codes or running hardware: the system should support standing up from this state by proper state and transaction management, logging, etc.
No, I can't call any of my works "well-architected" by this rule; but knowing and pursuing it made them "good enough", I think.


@ Lorand - good system design objectives but not a primary architecture consideration. Again, a system with a good architecture will show an ROI, which translates into it has/had value.


@Edward - true, I chose to list goals and not technology to achieve them, because I think the goals are abstract and usable for any system size, while the technology is not: using oversized equipment for a small task is not "well-architected". I know this very well because I always think too much, and had to realize that my approach simply does not fit well to a smaller task. Currently, I am testing the patience of a multinational company with my architecture...

To be more precise, I work on a fully configurable application structure including the GUI; only message-based communication between components which really have one and only one function. The core is so simple that GWT can take it to the browser, so I can use the same Java codebase on the server and the client (but have a transparent proxy logic for objects that can't work in the browser - OK, I have performance problems with IE, but that is my fault and I'll take care about it when I have time).
For example serialization requires 5 components in a row, but changing the stream format from JSON to XML requires only replacing the formatter/parser at the end :-)

I broke all deadlines while making it, and of course it is still in a "proof of concept" state, but already mighty beast. I love it :-)


@All - I hammer on the ROI for one reason, the question was - what makes a GOOD system architecture? 
We all have opinions about what is good, and that is good because one can learn from the opinions of others. However, our opinions can not be used to measure good, but ROI can. Therefore; I keep on going back to the only thing I know that can be used to prove of value was created - ROI. We should all work towards achieving that goal, and there are many ways to do it.


@Edward Perhaps it is not so obvious, but I totally agree: a good architecture must return the investment.

It means the whole design and development process is NOT focused on the delivery deadline and the mere list of required features, but the "long tail": how much it will take to keep that system alive in the upcoming years (and therefore: "refactor" even the original requirements in favor of the foreseen upgrades in the next decade), and in the end, make it a valuable source of information for its successor, and not a heap of smelly spaghetti (even though most system I knew already smelled before the delivery because of the last minute changes and patches...)

My 90-9-1 rule (from my point of view) is an abstract (but for me very pragmatic and useful) recipe to achieve ROI.

2014. február 18., kedd

On science and religion again

SURVEY: RELIGION AND SCIENCE AREN’T ALWAYS AT ODDS

I think it is hard to have a constructive conversation without clean subjects.

Separation of superstition from faith is the first important step. Understanding that "science" in our common sense is just another form of superstition (my favorite sentence is "I believe in science" - eh???). Understanding the fundamental rules, motivations and limits of a neural network that we all (yes, you too) use for thinking is a must. I don't see much of that neither on "religious" or "scientific" side. However, there are brilliant minds everywhere, like the Chan buddhists, Anthony de Mello, or Douglas Adams, who appears to be the greatest current philosopher to me (but I had to learn a lot to respect the Hitchhiker's Guide).

I myself got to the conclusion that

you are simply UNABLE to think rationally
without a consciously and responsibly CHOSEN faith.
:-D

If interested, check this out: http://hajnalvilag.hu/projects/TasteOfLuck.pdf