"Technical Dimensions of Programming Systems"

 lkedves

SEPTEMBER 1, 2023 AT 7:54 PM

Hi Jonathan,

Not having too much free time, I only skimmed over your article, and peeked into the site, nice one! I see a lot of similarity in the background, but I had an issue that you probably don’t hear too often. I think your abstraction is not deep enough. Although a generation younger, I have been building information systems from requirement negotiation to deployment and maintenance in the past 25+ years with a different core vision.

We cooperate with and via information systems that themselves are cooperating networks of various modules. This cooperation means learning and changing the state of the system through their modules and that goes from copying files, deploying existing modules, to changing their configuration or behavior – that is what we call “programming”. Thus, the “programming system” is just another module responsible to interact with the internals of a module, including itself of course. In this context, the ultimate module is the runtime that allows all the other modules work and cooperate in a particular environment: programming language and ecosystem over an abstract runtime or an operating system.

This approach allowed me to create a framework (the runtime and what I think you call a “programming system”) that I needed to implement other target systems (sometimes in hybrid environments). Testing it with your questionnaires:
Self-sustainability, 1: Add items runtime – yes. 2: Programs generate and execute programs – partial (as much I needed it). 3: Persistence – yes. 4: Reprogram low level – partial (metadata yes, runtime: code generation and build yes, “save algorithm as code” no). 5: Change GUI – yes.
Notational diversity, 1 Multiple syntaxes – yes (if you mean multiple programming languages and persistence methods). 2 GUI over text – yes. 3 View as tree – NO! It’s a free graph without such limitations. 4 Freeform – yes.

This was back in 2018, since then I learned that 1: the hard questions come after you have a working system on this level, and 2: a working prototype is an excellent cause to be rejected. This is too heavy for a pet-project, so I abandoned the main research and use bits and pieces in paying jobs (at this moment, creating a hand-coded knowledge base over the European XBRL filings for academic research). However, your research overlaps my playground, so here is another message in the bottle… 🙂

"Things I Wish I Knew When I Started Programming"

In short: you are right.

To add a few more words. 

I have been payed for coding, designing architectures and analysing problems for 25+ years. After 20+, I went back to the university to check if I am wrong or all the others - may call it "an inverse impostor syndrome". I got MSc, went on with PhD and during that time, I finally met with the fundamentals that explained why my mantras worked along the years. 

Like "structure eats code" means that we actually write information systems to understand a situation, learn to write a capable interactive model - or in academic terms, transfer our current knowledge to a Turing Machine only to extract a network of clean state machines, the process is called "refactor". 

Or: "I want to make my mistakes, not other people's mistakes" means the fundamental statement of information science: information is the noise. The things we did not know about the system. And we are there to make and understand different noise because from that knew understanding emerges. 

Etc, etc. Information science is an absolute gem, but it takes a lot of experience to understand... 

To be more specific and add Google compatible keywords. 

Understand the design patterns, they will work in any environment. Take the original Gang of Four materials. 

Check "Tracer Bullet Development" and other core ideas from The Pragmatic Programmer. That works. 

Listen to old guys, they did the same thing before the current "big names" appeared. Recommended: Bob Martin, Alan Kay. 

And if you think you know enough, you can start learning about the heavy stuff. Start with Bret Victor's Future of Programming lecture, https://youtu.be/8pTEmbeENF4 Then learn about those who he mentions: Sutherland, Engelbart, Licklider, Bush. They are the real deal.

---

The comment magically disappeared.

My very last talk with Frode Hegland

A Reply to Lorand

Loránd Kedves • 17 days ago
After closing with "over and out" the last time, it's my turn to be short(er).

- Yes, I write text. Sorry, I could not make a working Memex implementation alone in "spare" time. Yet, Dust Construct was quite close, even though I did not know about Memex when I built it.
- No hyperbole, pen and pencil Mr Hegland. A working Java application that managed its own knowledge graph, built data panels on the fly and generated part of its own kernel code from text templates. You shrugged it off every time saying you focus on text. Should I try it again? Challenge rejected. I trashed Construct for a good reason. You have seen that too.
- My statements are only one and essential step back from running source code. They should be specific for a person who can build the recommended Xanadu prototype. That is not me anymore.

---

Our conflict is much deeper than "text or graph". From my perspective, you are in the gang of shallow strong guys kicking me out and breaking my back again. I have no personal problem with you but I have a conflict with a system that you are part of, regardless of dreaming that you are against it. Just like in the Matrix:


You think you keep the heritage of Douglas Engelbart by repeating his words and mimicking parts of NLS "in high res". I think you break his dreamed revolution by "reaching for the low hanging fruits". I only wanted to leave a message at this memorial place with my first comment at your Tools for Thought post. I hope I will still be in a proper physical and mental condition to talk when someone understands it.

I close the circle repeating the Call for Mission Control... https://medium.com/predict/...

(links appear in darker grey on dark grey, hard to notice...)

Frode Hegland Mod Loránd Kedves • 17 days ago
Lorand, I am neither shallow, strong or against you. I simply ask you to be clear about what you propose. I have no problems with statements being the key unit of communicated thought. I simply accept that the interface between humans will always require interpretation since we don't even have fool proof objective access to our own memories.

Loránd Kedves Frode Hegland • 17 days ago
Can you accept that I can't explain my understanding and implementation of Memex (which is by definition NOT a "fool proof..." but the opposite), NLS or bootstrapping to you? It required me almost 30 years of programming, designing systems, trashing and reimplementing modules, and I can't transfer that network into your brain. If you really want to help, please just pass the link to people like Alan Kay or Ted Nelson, because they don't talk to people from the streets (and they are right doing so). That's all.

Frode Hegland Mod Loránd Kedves • 17 days ago
I do do have a working knowledge of the systems you mention but I’m glad you accept that you can’t just transfer that network into my brain. That is exactly the design issue, how to deal with that fact.

I’ll be happy to pass along a link. Which link is it you would like me to pass along?

Loránd Kedves Frode Hegland • 17 days ago
The Call for Mission Control on Medium. It seems that finding a link in your comment section is hard not only for me in the current color scheme... here it comes again :-)

https://medium.com/predict/...

... please add my apology for the harsh tone (and accept if for yourself too). Shouting for decades while everyone is going to the other way and complaining about it at the same time has its price.

Thanks!

Frode Hegland Mod Loránd Kedves • 17 days ago
I read your piece again and all I find is a tirade against the ways things are and a hope for something better. I can agree with this, it looks like an Engelbart C-Level task. I fully support this. If you are to get any support however, you really need to find a way to specify some specific things you feel can be done to actually move the needle forward in a specific and positive way. Do you have that written up somewhere?

Loránd Kedves Frode Hegland • 17 days ago
Following the links under "references" in my tirade may help...

The technical parts are: Dust Short, Software as a Data Structure, by the way a more current roadmap is in my introduction at Doug'50, look for DSL because all is about atomic core languages.
The civilization ideas: Taste of Luck, Open Source Civilization and Dawnworld (although this is 120 pages, it takes a strong motivation to read - but guess how much it took to write, then translate to English, etc.) I also moved some of my 800-some articles from my blog and LinkedIn to Medium, and had some interesting talks there in the past few days.

I know, less is more, but I really have spent decades on this...

Frode Hegland Mod Loránd Kedves • 17 days ago
I appreciate you have sent decades on it but what specific steps do you want to take?

Loránd Kedves Frode Hegland • 16 days ago
And finally understood the essence of this question.

First I thought "it should be obvious!!!" Join a team of brilliant people, make Memex together, build a transparent global network of knowledge, save the world. But looking at the actual environment, rationally thinking about specific steps, to my greatest surprise, I am not that sure. I mean, at all. My 30 years long "quest for the Holy Grail" has ended and I am free.

No further step is needed.

Thank you!

Frode Hegland Mod Loránd Kedves • 16 days ago
Your sarcasm is not useful

Loránd Kedves Frode Hegland • 16 days ago
It is already impossible for me to write a text that you can decode because I have a fundamentally different (yet quite effective) knowledge network in my brain and a very narrow communication channel. Do you really think I waste my time adding another layer of blur to that?

I mean exactly what I wrote. Our talk changed my core motivations. It feels much better now and I needed that.


Frode Hegland Mod Loránd Kedves • 16 days ago
What you write is the same as what we write do of course I agree. It is quite general and I apologise for trying to get something more specific and clear out of you. This discussion is not useful


Loránd Kedves Frode Hegland • 2 days ago
For the record...

We both know Xanadu's concept, demos and status, so I assumed we had a common ground.
I presented major architectural issues that I think caused its failure and recommended solutions.
I offered my help as a veteran architect and developer as part of my PhD research for free.
I mentioned that I did not want to work on it without talking about the changes with the "project owner", your friend, Ted Nelson. Because I could be wrong and did not want to waste time on something I did not understand correctly.

I don't know your professional background, but this is more than we generally have at a project kickoff in real life IT industry (at least according to my 20 years experience here). I did my best, it was not enough, happened many times before, no harm no foul.

Peace.

Frode Hegland Mod Loránd Kedves • 2 days ago
If you want me to forward an email to Ted I'll do that.

Loránd Kedves Frode Hegland • a day ago
No thanks. I have finally learned my lesson here and don't have more time to spend on a discussion that the partner judges useless. Happened many times, no harm no foul but this should be the last.

Frode Hegland Mod Loránd Kedves • a day ago
I don't think your work is useless. I think it's crucial to explore different options.


Loránd Kedves Frode Hegland • 6 hours ago
Sorry, you (and those sweet likers) still don't get it... :-D

I have literally spent 20+ years of my life on this work. I mean solving "impossible" problems in real environments, not "exploring options" in presentations. Its value and the Dunning/Kruger effect around it scares the hell out of me. This is where those ridiculous "non-specific tirades" come from.
I was not looking for these useless "come on!" messages (they only make me laugh and pissed off at the same time to be honest) but real cooperation with experts. The Engelbart community with you at the reception was my last resort.
I have spent many years trying to connect to experts, with zero result. You finally convinced me that I would talk to the hand all the time in "useless discussions". I am grateful for that. I will make it or break it alone. Business as usual.

Take care.

Frode Hegland Mod Loránd Kedves • 3 hours ago
I think you should consider how your moaning and insulting of people, coupled with a complete lack of demonstration of what you are talking about, only arrogant references, might be the reason why you have zero result. You work and ideas might be important and brilliant, or not. You make it a struggle for 'us idiots who are so stupid we don't know it' to tell.

Loránd Kedves Frode Hegland • 37 minutes ago
For the record again:

I tried to show you a working garage-Memex. I used it as a thinking augmentation tool to design both itself and a real ETL web application. I have built it before I learned about Engelbart or Bush. You had no time and were interested in text. Fine, but to me far from "complete lack of demonstration".

I came back only because you started drawing graphs and wrote about Tools for Thought. That's what I do. And yes, it takes serious effort to understand, otherwise Google, Microsoft, IBM etc. would have done it. That can make others feel bad. Sorry, that is not my problem, making it is much worse.

I can show things but can't make people see them. After a long time that made me impatient and the whole thing became even more hopeless. That leads to our shared conclusion: I should stop talking. But I saved your excellent summary on me for later reference.

Thank you.

Text vs graphs in knowledge management

Why work on Improving our Tools for Thought?


Loránd Kedves • a month ago
Dear Frode,

we had our short journey together but to me it seemed you focus on text and existing toolchain, while my obsession is what I call today (thanks to Gyuri!) idea projection: the process of direct interaction with the network of knowledge, and the requirements of an infrastructure that allows the continuous morphing of that network (that is merely impossible with what we have today).

Now the development is on hold due to personal financial and health issues, but I still write texts :-) sometimes. I guess you may like it... https://medium.com/@lkedves...



Frode Hegland Mod Loránd Kedves • a month ago
Hi Lorand. Yes you are right, I am focused on text, I feel that external symbols are a powerful way to manipulate our thoughts and communications. If you can find a way to directly interact with knowledge, that would indeed be amazing but I am a text guy and that is outside of my area.


Loránd Kedves Frode Hegland • 25 days ago
OK, but then please explain me why you put a graph to the top of the article as an illustration of Tools for Thought? That is not text, you can't use a text editor to author it (yet another markup or source language IDE is not a "text editor", but manages hidden properties assigned to text fragments).

My problem with your approach is that as long as you put an equal sign between thoughts, symbols and text, you can't even see the point: the also morphing language and syntax required to transfer (serialize! so no graph here) your knowledge to text. A real Tool for Thought (which I have already created and used to improve itself before learning about Doug, but was too deeply locked to one platform) should include changing ALL, even the rules it uses to fully describe itself. That is missing since Bush described and Doug partially implemented Memex.

Focusing on text (writing yet another editor) is a dead end street, as example see Ted Nelson's Xanadu progress in the past 50 years. By the way, I don't see the fundamental improvement in Liquid over Xanadu. What is it?


Frode Hegland Mod Loránd Kedves • 25 days ago
I don't think text needs to be in a column, I guess that is where we have different perspectives. I'm sorry you feel my work is a dead end street and no, I am not trying to go down my good friend Ted's route, it's a different direction.


Loránd Kedves Frode Hegland • 19 days ago
> I don't think text needs to be in a column, I guess that is where we have different perspectives. I'm sorry you feel my work is a dead end street…

On your level of extreme oversimplification

Not exactly.

On my level of TL;DR

I have been writing source code for living for 20+ years. I always hated to bury my understanding of a task into serial text of the chosen programming language and libraries. To me the real work is building the structure and interaction of components. That can be wrong for the first time, this is where design patterns, refactoring, Agile methodologies come in. They are as natural to me as breathing, in fact I used before learning about them. I created always better tools to support design and automatize coding until Dust Construct that I used to think about and build itself. At this point I first met Engelbart's ideas and realized that this is Augmenting Human Intellect.

I was already at the border of "machine text" and plans of "human text" and asked if thinking in general can have the same problem as source code. I have found Tim van Gelder, a philosopher who refers to Engelbart, derives argument maps from text to represent the real content. He says that a communication error is the difference between the knowledge graph that the reader decodes from the text and the one the sender wanted to transfer. We need tools making these graphs transparent to decrease errors caused by the weakness of text. I found Neil Postman, who explained the dangers of using modern information systems without understanding how much they change our thinking. And many, many others.

I say text is outdated because of the
- need of serialization,
- lack of exact definitions and structures,
- missing ability to follow morphing background.

I knew that you did not listen when I tried to explain this. Your not-even-wrong summary adds those months to the long list of bitter mistakes of mine. Thank you for making it clear.

> … and no, I am not trying to go down my good friend Ted's route, it's a different direction.

Sure. Xanadu is not only ahead of Liquid by 50 years, but almost solved the problem. Yet, Ted made some fundamental mistakes in the Why? (Vision) to What? (Architecture) translation. These are:
- The “atom” of text is NOT the character, but the statement. References exist among statements, not character ranges. This allows changing other statements or translating the text without breaking existing references.
- Store statements by using combined IDs: internal primary key + commit ID. You can change statements later and save them by the same internal but different commit ID. You keep the original references and can follow updates on Xanadu level.
- Trash EDL, understand that any given text is a hierarchy of statements, either local or remote. You need to store the links in their natural bidirectional form, and build the text from a root node to display. The statements are not locked into their owning text, but a flexible cloud open for reuse. By the way, this is the essence of ZigZag database.

This Xanadu prototype is a 6 months spare time project for a developer with my skills. I started it using Hungarian laws as data source for my research, but trashed due to the lack of support and feedback. I also close my PhD because I “lost” the time needed to get the required publication credits.

"Your good friend Ted" was quite close to see his 50-year-old dream working TODAY, beyond a quite weak 3D framework demo of squares with custom texture and colored rectangles. But, he does not talk with strangers on his YouTube channel and has you protecting him from the chance.

If anything, you’d better be sorry for this instead of my guessed feelings.

To quote the Oracle from Matrix:
"I'm sorry, kiddo. I really am. You have a good soul. And I hate giving good people bad news. Don't worry about it. As soon as you step outside that door, you'll start feeling better. You'll remember you don't believe in any of this [Lorand] crap."

Over and out.


Frode Hegland Mod Loránd Kedves • 18 days ago
Lorand, I have replied to you here: http://wordpress.liquid.inf...

Knocking on and on...

On LinkedIn again...



Lorand Kedves
That sounds great, but here is my billion dollar question:

How do you enter "the valley" with a really fundamental idea (and working prototypes by the way) that would take too much time to understand? When everyone turns away after the second talk saying “Never invest in a business you cannot understand”?

Careful, this is where I failed with you as well... ;-)

Peter Kovacs
You cannot enter any market, even your local one with an idea that is not easy to understand. That's why good timing is a crutial part of success.

Lorand Kedves
Peter Kovacs I don't talk about "entering the market with a disruptive strawberry flavored chewing gum", but core technology.

I went back to the university as student at 42 to check if I am lunatic. Instead, I have learned how my research is connected to Neumann, Turing or Shannon. I had to ask my friend to be the supervisor of my thesis because for him it was OK that he, the teacher does not understand.

I can tell you WHY the Neumann architecture was ground breaking, WHY the most brilliant programmer, Charles Simonyi works on Intentional Programming behind closed doors. I can show you why the elephants work on PaaS and what they do wrong. But of course it takes time and effort to understand, which is not popular today…

Of course, our last startup failed - the gods of IT are used to talk, not to listen. I am unemployed. At least I have a few months to focus on my studies, thanks HR guys :-)

So yes, I really respect George's message, and I think you can't blame me for being shy, or unaware of the cultural barriers. I only ask for your advice: What do you think the "good timing" is for a new Neumann architecture - if you don't want to see it in the patent portfolio of a giant? :-D

Peter Kovacs
Lorand Kedves there are chances you are the type of guy whose invention will be only discovered and appreciated after the death. Or not :)

Lorand Kedves
Peter Kovacs I asked a practical question, you gave me your ultimately depressing opinion (which is my direct, personal experience of 20 f*cking years of guts out work that you can't even imagine, that's a cultural barrier, bro, but I am not Eminem, so stop it now...), then "fixed it" with a wink. Should I inform you that this is in direct opposition to George A. Tilesch 's message, or can find it out alone? :-)

But let me answer in your style.

I am 43, so (if you dare to look around for a second with open eyes) I will likely last longer than the illusion you have about the world. See here for a very sort introduction, which took me weeks to create but you will close it within a minute if you click it at all, because "focus is important" - take my word: _direction_ is the first... https://goo.gl/om22gB

But if you are right, guess what "they" will think of you, personally? So, you queue up to a relatively big and illustrious group of people who pray not only for my early death but also to be completely wrong, while I, quite naturally, work for the opposite. Why are you my enemy?

For the closing wink, quoting John Cleese: "Was it funny?" An "afterwink": was it meant to be? :-)

Thanks for your time.

Knocking on Heaven's Door :-D

At OpenAI gym.

May 14 08:31
I would ask you a silly question: what is your definition of "intelligence"? No need to give links to AI levels or algorithms, I have been on the field for 20 years. I mean "intelligence", without the artificial part, "A" is the second question after defining "I". At least to me :-)

May 14 21:47
@JKCooper2 @yankov The popcorn is a good idea, I tend to write too much, trying to stay short.

@daly @gdb First question: what do we examine? The actions (black box model) or the structure (white box)?

If it's about actions (like playing go or passing Turing test), intelligence is about "motivated interaction" with a specific environment (and: an inspector who can understand this motivation!). In this way even a safety valve is "intelligent" because it has a motivation and controls a system: it is "able to accomplish a goal". Or a brake control system in a vehicle, a workflow engine or a rule based expert system.

However, white box approach: how it works is more promising. At least it enforces cleaning foggy terms like "learn", "quicker", or how we should deal with "knowledge representation", especially if we want to extract or share it.

In this way, I have starter levels like:

• direct programmed reactions to input by a fixed algorithm;
• validates inputs and self states, may react differently to the same input.

So far it's fine with typing code. But you need tricky architecture to continue:

• adapts to the environment by changing the parameters of its own components;
• adapts by changing its configuration (initiating, reorganizing, removing worker components).

So far it's okay, my framework can handle such things. However, the interesting parts come here:

• monitors and evaluates its own operation (decisions, optimization);
• adapts by changing its operation (writes own code);
• adapts by changing its goals (what does "goal" mean to a machine?)

At least, for me artificial intelligence is not about the code that a human writes, but an architecture that later can change itself - and then a way of "coding" that can change itself. I did not see things related to this layer (perhaps I was too shallow), this is why I asked.

May 16 06:10
@gdb Okay, it seems that my short QnA does not worth serious attention here. I have quite long experience with cognitive dissonance, so just a short closing note.

Do you know the Tower of Babel story, how God stopped us to reach the sky? He gave us multiple languages so that we could not cooperate anymore. With OpenHI ;-) this story may resemble the myriads of programming languages, libraries and tools - for the same, relatively small set of tasks, being here for decades. (I have been designing systems and programming for decades to get the pain of it - see Bret Victor for more.)

So my point here: Artificial intelligence is not about algorithms, python codes, libraries, wrappers, etc. that YOU write and talk about. All that is temporal. (And by the way, AI is NOT for replacing human adults, like Einstein, Gandhi, Neumann or Buddha. It is only better than us today: dreaming children playing with a gun. hmm... lots of guns.) However...

When you start looking at your best codes like they should have been generated. When you have an environment that holds a significant portion of what you know about programming. When it generates part of its own source code from that knowledge to run (and you can kill it by a bad idea). When you realize that your current understanding is actually the result of using this thing, and that you can't follow what it is doing because you have a human brain, even though you wrote every single line of code. Because its ability is not the code, but the architecture you can build but can't keep in your brain and use it as fast and perfect as a machine.

By the way, you actually create a mind map to organize your own mind! How about a mind map that does what you put in there? An interactive mind map that you use to learn what you need to create an interactive mind map? Not a master-slave relationship, but cooperation with an equal partner with really different abilities. I think this is when you STARTED working on AI, because... "Hey! I'm no one's messenger boy. All right? I'm a delivery boy." (Shrek)

Sorry for being an ogre. Have fun!

Dust state

https://www.linkedin.com/pulse/critical-new-role-system-simplification-specialist-roger-sessions

Roger, I could not agree more, with one note: the definition of "mathematically". I have not the best feeling about that term, ending in complex expressions with weird characters. It is quite easy to frighten away programmers like me with that, even though I know that with proper handling, that is the best way to express something. And even though at one of my workplaces, they always told me "this is not university, it has to work!" But they gave me time to make it work, and used it. Good old days.

I think the situation is similar to where we were before John von Neumann.

At that time, reacting machines were made as custom logical circuits, and they had to fight for every second with the slow and thirsty hardware. Then he came with his brilliant essay saying "don't solve the problem as a whole, but separate the components that are required for a reliable IT solution, make hardware for that, and instruct them to solve the problem."

That must have looked stupid for all engineers: make it even slower and more complicated? But their actual designs simply could NOT be verified, optimized, not to mention being built into chips, they had to build everything with their hardware tools, and their knowledge was buried in the metal. Solving real problems with direct building was so complex itself that higher level constructs were simply over the horizon. Simply put: the complete approach was not industrial.

Replace the "hardware" with "big software components, toolkits, platforms", and you have the IT world today.

Everyone hacks together big "systems" from big and custom components, there is nothing that you could rely on because all is done individually by someone, upon (and locked to) constantly moving tools, chasing constantly moving targets. There is no reusable terminology and actually working toolchain that you can use to express your needs on a higher level, above being locked to your tooling.

That part should encapsulate the complexity of atomic segments of problem solving, and offer the solution in a global, uniform, usable and efficient way. Nonsense. Unless you spend 20 years on learning to see the structures instead of the solutions, and for example, separate the needed code from configurations and repetitions. Like me. Unfortunately but naturally, this separates me from "the rest of the world".

I can show you a prototype of a system that contains its own definition, and much of my knowledge about programming. It can generate its own Java sources and projects, but is "theoretically" independent from the language itself. I had to create it because its design is so complex that the only way to make it work is to let it actually execute its own configuration.

Of course, it is not different from the Wright brothers' "thing". It is just a bunch of hacks compared to the super-sophisticated trains that we use to "solve problems" today. Okay, okay, it can fly, but who cares? Or who would waste time to understand its weirdness? How does it carry anything? This must only be a toy, huh?

The first "real" question is: how do you sell it? Well, if you understand what it means, all big names would die to have it and lock it in a box, because it simply changes the IT business. The next is: does this ever solve a "real" problem? Come on, your "real problems" are BORING, and exist mostly because of short vision, unforced design errors and mindless cult-coding... ("... you are so rude, Sherlock" ;-) )

So, regardless of the trust I received recently, it's on the waiting list. I now write a frontend to a webshop for the monthly wages and watch "transit gloria mundi".

Can a good GUI framework be created?

LinkedIn: Isolation Business layer from UI - Does it make sense?

Dmytro Kurylovych
Lorand, I believe there is a big role of UI controls and common classes of pages in your framework.

It is hard to perform efficient coding without commonly accepted base functionality - which is a foundation of lexicon, requirements and semantics.

This lexicon defines what things ARE and what things DO - primitive or composite. Noteworthy, elaborating those concepts - we will find sustainable communicational (integrational) schemes between components - which are more that the sum of parts. You will undertstand me if you created and supporting UI framework more than 2 years.

This is a kind of evolution of knowledge - when long time is needed to realize fundamental blocks from many prospectives, and then - begin realizing their efficient communication, tuned explicitly, or implicitly on top of characteristics of component.

I don't believe in efficient conceptions. Any MVC is nothing without a stable set of components and working samples. Absolutelly agree with Matt about the boundary, untill there are still many efficient architectures which break that rule, but provide more security and less boring code. Good architecture helps to achieve compromis, but unfortunatelly - conceptions are very general and they lead to tonns of unstable code if practiced literally.


Dmytro, I fully accept your concerns. I have worked a lot with GUI in C/MFC, Java/Swing, J2EE with JSP and some web frameworks, ObjectiveC/Cocoa, C# whatever (that was short). I did not like those frameworks (yet, Objective C is "almost there") because they all focused on implementing the requirements in a system - and again asked the wrong question. The question I offer here is: What is GUI? Why we create it?

The answer is: we want to let the user interact with data. Not "creating and handling selection change event" or "disabling a control". Talk with data. In this way, the answers are more simple:

• we have a finite GUI control set in our environment (like Java Swing, GWT or ExtJS). Create wrappers that listen to their OWN changes; handle their OWN parameters (can load from a configuration and apply them); and also their OWN states (visibility, enabled, style);
• we need a data layer ABOVE the actual data (forget about using POJO here). I use plain old HashMaps hidden behind an API. It is 1: fast enough on these machines, and 2: the core implementation can be improved without touching the using codes if (1) is not true anymore;
• open this data layer to the GUI framework and the business logic you need in your current task
• we need an identification and listener mechanism that transfer control and let them talk to each other
• and not to forget: a shared status reporting mechanism that allow each component to "complain", which the GUI can display and the Process can use (abort on error)

This is really complex, I admit. However

• should be done once and in the long run, worth the time (you only need to implement the actual business logic in all the next systems)
• has nothing to do with the actual business needs - or in other words, these are the business needs to build a GUI framework :-) (this is THE recipe to create stable code: focus on exactly one thing in a component, so the more you know, the less you change)
• does nothing else than you would do in any system GUI (only the weird stuff is concentrated into one component instead of typing them in all the time), so this far is as efficient as direct coding
• yes, I use maps instead of objects, which is of course, slow. What I get? No getter/setter functions with listener management, no long, framework-bound "Control" codes. In fact, zero direct GUI code, because the required business logic interact with data, and not GUI controls.

That ten years old framework had an XML based configuration, an editor to build the GUI, renderers to create interactive Java Swing for fat AND Javasrcipt/HTML based web clients, and PDF output - not to mention that it run without GUI when we needed mass update from text files. We used other configuration to connect business logic on the data the GUI worked with - it got control on change, and the GUI got control to update its content. However, actual GUI coders did not know anything about it, because they had to

• create the GUI configuration, put it into the system and it appeared (of course it was modular, so could use pre-created fragments);
• select the required standard logic components (dates, users, filing information, etc) and connected to the data items, so they have shared, tested, stable Ergonomy/Consistency elements, still without coding;
• create the custom Ergonomy/Consistency logic. Still without knowing anything about the actual GUI framework, because they interact with data, not the GUI (or other internal, Process components)

... and they did not care about storing the data, make their stuff appear in common lists, be printed to pdf or mailed to the mentioned people because that was in the Process logic layer. The greatest problem I heard of was that working with it is boring. Considering the dark magic behind, I take it as a compliment :-)

Layers of business logic

LinkedIn - Isolation Business layer from UI - Does it make sense?

Great timing - I had spent quite a few days with meditating on this question lately, and I am working on a framework to support my approach, so forgive me venting this here.

First, a bit of eastern-like statement: If the question is right, the answer feels natural - when the answer is foggy, perhaps we should ask a better question. I offer the following "better question": why we create business logic? Can we classify logic layers?

1: Execute the planned actions (control equipment, modify data, etc.), this is "Process";
2: Protect the system from malfunction or corrupt data, ensure a coherent state, this is "Consistency";
3: Help the user do her/his tasks in a comfortable environment, provide all information to make the right decisions quickly, this is "Ergonomy".

To me, this classification makes the answer quite obvious.

Process is in the core, it uses all the sensitive resources (databases, hardware, etc.) directly following the actual commands with the given data. The data must be clean and error-free at this point, only system errors can occur in forms of technical problem reports (system availability, low level DB exceptions) that requires the attention of the support team immediately, and should give an "internal error under investigation, come back later" message to the end user.
In a heterogeneous environment, this is the backend, server, the center. In my framework, this is a fully configurable network of hardcore Java components.

Ergonomy is the other end. It is absolutely essential part of the system, but the support team does not care about its messages: it translates the notifications and actions of the other layers and reflect to changes (notification lists, field states and value sets, calculated value updates).
If at all possible, it should run on the client only, and use minimum amount of the valuable server resources. In my framework, this is written in JavaScript, so it can run in a browser or in a modern JRE for fat clients.

Consistency is in between. First, it MUST run on the server in the same synchronized transaction with the Process, to reliably protect the sensitive elements. It might use heavy-weight components and sensitive data that should never go down the line to the client. At the same time, some parts might run on the client side to save server resources and provide smooth client experience. Consequentially: this is also JavaScript, can partly go down and run at the client side to increase performance, but must completely run inside the Process transaction.

Of course, this is my ideal approach and is under construction, but so far looks good. My 2 cents.



B. D. "Beck" Beykpour
Lorand, I'm thinking of automation software that I've work on. I think you're model does fit here as well. I've often thought that consistency and safety had to be implemented at two levels. First, it should enforced at lower levels to ensure that nothing goes wrong. However, it often has to be reflected in the UI as well, whether disabling certain conflicting commands or informing the user about a possible conflict in their requests.


Beck, I have to admit that there's a catch here: this approach is less than 2% of that framework, and without the rest 98% it is merely a fairy tale...

With a simpler approach however, you can still make this work. You have to create "abstract smart widgets", a separate data layer, and a logic layer in between. In this environment

• the widgets are responsible for handling their own events and transfer them to data change commands and events for buttons
• they also "listen" to data changes, and that mean both their 1: actual value, 2: the value set for lists, combos or tables and trees and 3: decoration like tooltip, visibility, activeness, style
• logic can sit behind data and listen and react to the data change. Ergonomy works with the decoration, Consistency with the valuesets, availability and status report - Process is what you don't see at all
• and you need "light bounds" between them: access controls and data by string identifiers and should support following paths. ObjectiveC does a good job with it, in Java it's a pain.

Well, even this simpler approach takes a lot of time, for me about 1.5 years to run a public national administration system, ten years ago (but afaik it still runs and gives work for a hundred developers, I felt forced to leave after creating it) :-)
In practice smaller scale apps do not worth such amount of effort, but still good to keep this segmentation in mind. So again: sorry for venting.

Artificial Intelligence 3 - The Tower of Babel

Dear Jordan,

thank you for the response, I really don't want this talk to settle down, but there is a human part in it... I could write those long articles because I have flu and could not sleep after 1AM. Your question about religions is absolutely important and deserves a long response, this is just a short note, sorry for that.

I see religions from a completely different point of view: they are meme constructs that survived during the evolution of the human civilization (which is a very close replica to the biological evolution, with dirty fights among rigid rules), and proved to be able to form a community from human individuals. They are like X-Ray images of strongly-bound human groups from different angles, not "obsolete", in fact the opposite: they contain patterns that can build a super-entity from human beings.

This super-entity: a conscious global human race is the true counterpart of the ASI - in fact, they can't live without the other. We, human individuals are only cells of this global organization - but the organization itself can't emerge if we are not conscious about this fact. I used myself as an example describing how much I am integrated into this system, and that my existence is just meaningless without it. This is true for all of us, but most of us "don't like it".

If we create the ASI while we are separated individuals, our childish approach (to life, the planet, to other humans and to ourselves) will not be able to control it, and inevitably destroy ourselves. The human super-entity can't even exist without its nerve system and brain: the ASI that can hold our accumulated knowledge and can support and connect us with it, let us use and make better decisions, do clear and efficient communication (compared to the current info-guano).

We also disagree on the quality of today's civilization.
Yes, we have million times faster and smaller computers, and can count the hair in Shrek's ear - but the world became even less transparent by them.
We learn the rules of geometry, physics (well, most of us quite hardly) - but where is a Thales today who would find them out by himself?
True, today's literature, films, etc are gigantic, we count thousands of years, millions of light years - but does that compare to the wisdom of Tolstoy, Lao-ce, Jesus, or Douglas Adams?
Techno "music" can be computer generated, and with proper "medication" it can give you the "trance" - but does that compare to Mozart who can break your soul into pieces and build it again (if you can listen and are sober), or the meditation techniques and koans of Zen Buddhism?

I have found my favorite statement about this right here on LinkedIn by Patrick Joiner:

"I suppose one way for a machine to pass the Turing test is to wait until the quality of actual human conversation is so bad that a bot could be an improvement. This seems to be happening here."


I have written a short summary about the "rational part" here: http://hajnalvilag.hu/projects/dust/dustShort.pdf
and about my (naturally irrational) faith in my book, page 67, "ready to fly":
http://hajnalvilag.hu/books/MondoAurora_en.pdf

... but if you excuse me a joke here: if you can really read the Bible, this is the hidden wisdom behind the story of the Tower of Babel. If you read it like a story book: God disturbed the talk of men to stop them reaching the sky.
But if you read it like a cookbook: to reach the sky, you must talk the same language. That "language" is the accumulated knowledge of the human civilization in a globally uniform and usable declarative structure, and "talk" is our current global, simultaneous communication infrastructure.

This is the difference between hordes of isolated ANIs in the hands of selfish, short-visioned individuals, compared to the ASI, in the hands of a strongly-bound human race.
The latter is the only way to reach the sky... or to be honest: essential to survive on this planet that can collapse without help.

Artificial Intelligence - 2

LinkedIn is a good place!


Jordan Hughes
For a different perspective, see my blog piece in Artificial Times: "There's no cause for alarm - just be quiet and drink your Kool-Aid," written in response to an article by IBM researcher David W. Buchanan published on Feb. 6 in The Washington Post: http://artificialtimes.com/.

Dear Jordan,
I have checked your profile, and saw that we have common interest areas, although different perspectives: I think we (human common sense) have no rational view even for the next 10 years, not for 10000... maybe you would like this lecture of mine.

Regarding AI, congratulations to your very professional blog, it was a great pleasure to see – I hope one day I will appear on it :-) I have checked some of your links, and I think our perspective is not that different.

It seems that we agree on the existence of danger emerging from IT systems. The difference is that I focus on the human side: we should not rely on any ethics from an IT system, we only get back what we put in. This is independent from the speed and performance of the system, and already causes huge problems. With more computer power, it will surely increase, and this is lethal to our civilization unless we change our fundamental motivations and attitude.

We also agree on the fact that the lack of fundamental definitions (intelligence or consciousness) makes objective evaluation and discussions fuzzy. However, I have found practical definitions that I actively use in my system design and programming tasks. I plan to write about them soon.

I still disagree in the “automatic emerging” of a conscious entity by any definition, from the mere performance increase. I think for any improvement it is fundamental to find a different IT representation of our knowledge than the current business-driven approach, separated by the various platforms, languages, formats and toolkits. Without a completely transparent and reusable knowledge store, the more an IT system “knows”, the greater amount of custom source code and configuration is created, therefore the system is less reliable, flexible and interoperable.

I think this is the key step to forget the current hordes of closed ANI systems: when their data structures and messages become transparent, they can form an interacting mesh network. I think this is the base of an ASI, and its “personalized agent”: AGI. No wonder: the focus of my research is a transparent knowledge representation, I have quite advanced working solutions – and not less ambitious plans :-) [acronyms: Artificial Narrow/Generic/Super Intelligence, as explained here]

The other difference is how we see the coming massive power of our global IT infrastructure, and the articles I read show the same fear that I mentioned in the “physical power” analogy. Before the “machine revolution”, people were afraid of their imagined “inhuman” power – today we are not afraid of a crane, an airplane or a power station, although they are much stronger than those people could imagine (the difference between the person from 1750 and 2015). I think the fundamental difference is a paradigm shift: we don't have to compete with machines where they are better, but can form a kind of symbiosis, combining our true human values with their not less true artificial power.

This symbiosis already works for me: a part of my mind is not in my skull, but exists on the Internet and the word processor, design and development tools I use. Without them, I could not do my job at all. A system that I build “has” all my knowledge that I collected in months, but some concepts literally took me decades to achieve. My system contains and uses it all in every single moment, does more in a second than I could do in weeks (but I would make tons of mistakes). Is this system smarter than I am? Of course. Is it dangerous? Maybe, in wrong hands – but not because it is smarter or faster. It is me who adds the “human part”: the roots in ancient philosophy and religions, and the motivation to seek for the benefit to our civilization.

Yet another fundamental difference: immortality. I guess someone envisioning human immortality does not understand the most fundamental aspects of life and human thinking. The very definition of all life forms contains death as the rule of the game – and the most fundamental motivator of human thinking is knowing about and hopelessly struggling against this concept. There is a clean analogy of “biological immortality”: cancer, and this analogy perfectly describe our attitude and its result on our living environment. For human beings, this is the key improvement required to enter the new era: understanding and embracing personal biological mortality – and “memetic”, informatic immortality: our ideas, or practically anything appearing about us in the IT infrastructure will be preserved by it and become “timeless” compared to our limited and time-bound existence. Another example: space is not for human individuals as the current “spacey-tales” suggest, but for the whole human race, through lots of personal sacrifice, pain and death. This is the only way our civilization improved in the past and will improve in the future.

We'd better grow up to this before it gets too late.

---

Alexandru Grosu
It is time to implement the four laws of Asimov into all the AI APIs. Now, it is still safe.

The most fundamental statement regarding the AI is that there is no way to implement ethic concepts in algorithms and no reason to assume that a software can be ethical. By the way, Asimov himself gives a very foggy explanation about how the laws are burnt into the pozitron brain.

I think the explanation is simple: his robots are not "AI previews", but represent human beings with greater power or higher level mission. His stories show psychological effects of coping with greater tasks or fundamental ethical laws, racism, fear. He talks about how our intellectual, political or financial elite should behave, what laws they should follow after they have separated themselves from the simple burdens of human life (the lower section of the Maslow pyramid if you like).

I also think that his laws don't work, because they are negative or conflicting: “may not... except”, just like “don't kill” or “don't be evil”. They freeze a conflict between our motivations, rules and interests: it would be profitable to be evil, but I should not be – but hey, what is evil anyway? and how far can I go without being evil? should I lose if my competitor is evil?
I think positive laws are harder to follow but they focus on the path where you should go instead of walls that you should not break. I see this improvement between the 10 commandments “don't”s and the simple “love your fellow as yourself” (if you love someone on the same level as yourself, there is no motivation to break the commandments...) My actual favorite is the Shotokan Dojo Kun (which is absolutely useful and practical in everyday life as well):

• Seek perfection of character
• Be Faithful
• Endeavor
• Respect others
• Refrain from violent behavior

And again: this is not for the AI, but us who create and use it. We failed on this mission with the machines (“officially sacred” weapons cause the same suffer), and true: with AI, this failure can easily be our last one on civilization level, but not because of the “evil AI” but our own irresponsible, childish attitude that now meets with Darwin's law.

We are not "too big to fail", but "too big to survive this failure"...

Kell-e félnünk a gépek lázadásától?

Index: Kell-e félnünk a gépek lázadásától?

HAL. Skynet. Mátrix. Johnny Depp. A scifi-történelem tele van öntudatra ébredő, gonosz számítógépekkel, amik (akik?) az emberiség életére törnek. A mesterséges intelligencia azonban a valóságban is egyre fejlettebb, így nem csoda, hogy egyre többen félnek tőle a mozin kívül is. De ha már Stephen Hawking vagy Elon Musk is aggódik, akkor érdemes megnézni, miért van igazuk, és miért nincs.


Kedves Dániel,


ha érdekli egy önmagát szakértőnek tartó ember véleménye: NEM.
A használó felelőtlen, gyerekes hozzáállása az igazi veszélyforrás.

Az előző ipari forradalomban a divatos rettegés a gépek fizikai erejére vonatkozott: jönnek majd a hatalmas, elpusztíthatatlan gépek, és agyonvernek mindenkit.
Aztán kiderült, hogy a gépek erősek, gyorsak, sőt úszni, repülni is tudnak - de maguktól nem vernek agyon senkit. Viszont mindegyikre lehet bombát és gépágyút tenni, és emberek megölésére használni őket - de ez nem a gépek hibája, szerintem.

Most az intelligens gépek a következő téma, és olyanok vitatkoznak róla, akiknek tudtommal nincs pontos definíciójuk arról sem, hogy mi az intelligencia, és mitől nevezünk intelligensnek egy gépet. Nos, nekem van.

Én húsz éve írok számítógépes programokat, tervezek rendszereket. Mások számára felfoghatatlanul sokat dolgoztam azon, hogy "humán agyam" képes legyen gépi színvonalú struktúrákká alakítani a meglehetősen amorf emberi világ egyes szeleteit. Ma egyfajta szimbiózisban élek a számítógéppel és az internettel: a tudatom egy része nem az agyamban van. Nem tudom fejből az általam használt programozási nyelv minden elemét, ezt az internet és a fejlesztőkörnyezet pótolja úgy, hogy nekem csak a valóban emberi részre: a struktúra megalkotására kell koncentrálnom. Az általam írt program a tudásomat reprezentálja, vagyis rögzít egy sok hónapos, éves fejlesztési folyamat során felépített gondolati rendszert, majd ezt nálam sokkal gyorsabban és megbízhatóbban alkalmazza. Az alapstruktúrát pedig szó szerint húsz éve fejlesztem.

Intelligensebb az általam alkotott rendszer, mint én? Igen, természetesen: mindent tud, amit én megtudtam a problémáról, gyorsabb és megbízhatóbb. Ha úgy írtam meg, a környezeti feltételek változásához is képes alkalmazkodni, tehát adott pillanatban olyan állapotban van, amelyet nem én találtam ki.
Kell félnem tőle? Fel sem merül, ahogy nem félek egy autótól mert gyorsabb, vagy egy darutól, mert erősebb nálam.
Veszélyes? Igen. Hatalmat adhat olyan emberek kezébe, akik nem képesek azt kezelni.

Gondoltam rá, hogy írni kéne egy könyvet erről, de egyelőre csak az első fejezetig jutottam. Pedig lenne mit hozzátenni :-)

Üdvözlettel

  Kedves Loránd

Contact experiment - Professor Noam Chomsky

Dear Professor,

Sorry for disturbing you, but I always throw messages in bottles towards really interesting people, being one of my a role models you are an obvious target.

Here is a short meditation about the similarity between digitization of computers and patternization of our brains (being a successful SW architect for 20 years gives me a bit of background):
http://hajnalvilag.blogspot.hu/2014/10/coping-with-infinity-digitization.html

If you liked it, here's a longer lecture about our thinking and the world: http://hajnalvilag.hu/projects/TasteOfLuck.pdf

And here's a book, if you are still have patience: http://hajnalvilag.hu/books/MondoAurora_en.pdf

Best wishes
  Lorand Kedves (42)

---

Dear Professor,

it is already an honor that you responded so quickly. Respecting your valuable time, here is my elevator pitch, and I will not further disturb you until you request it.

I am a software architect, worked with 5 - 200 colleagues, including international companies from data mining/AI to government data management systems.
I concluded that IT industry has forgotten about the Neumann principles; it focuses on the required functions instead of the structure of a system. Solving a task for me is to create a language (terms and activities) in which I can "explain" the computer what to do. Today, I "think together" with my framework: register my findings in a meta-layer, because I have written the codes that can understand and execute my plans. This system is "more intelligent" than I am: my brain can't hold and use reliably all the knowledge that I have collected about the task.
My final goal is to use the same descriptive tools to the "executor" itself, and make it completely runtime and language independent (and it is not too far now).

IT is extremely important, because it is the front line of today's constant war. First, people killed others to enforce their own will. Later, they conquered and enslaved them. Even later, they created laws to keep others in their order. Today they make others believe in a vision of the world to remote control them. We have the IT tools (personal endpoints and global, real time transport network) to show the real world to people in the same way as now they are blinded by, and blind themselves with false dreams and empty chatting.

I am confident that this is inevitable consequence of the fact that our economy concepts (money and natural human motivations) are not able to describe and organize the operation of a global and synchronous community: the human race. We are in the very last moment to change this path. I think I know how to start it, and I think, you are one of the few who can evaluate my ideas, because we seem to use compatible patters, and thought a lot about languages, ideas, reasoning, brains and communities.

Thank you for your patience.
  Lorand Kedves

Implementing workflows

LinkedIn: What are the best choices for implementing UI page flows and Batch work flows

When I face this task, I consider the following (always appearing) tasks:
 - You MUST do the same when the process is done on a GUI or in a batch processing of thousands of cases - so you MUST use the same codebase. Period.
 - The flow (the states, paths, control conditions and addressed objects) WILL change in time. The more business logic you write in code (and almost alike: annotations, magic XMLs of various toolkits) the sooner you will get hurt. Not to mention that sometimes you are asked to open the flow for editing to customer administrators.
 - The users will NOT read the manuals, so they will get angry unless they get all information: what are the available options, what they mean and why other options are not available.
 - Unfortunately, sometimes external toolkits also change, so if they appear in your codebase, you have to follow them (and in worst case, your users will also have to adapt to the change).

Now to the better news:
 - A workflow is a mesh network of State and Transition "objects". It is really easy to store this network, even in simple JSON if you like (each state has unique ID which is referred in the Transitions). That means you can send it to a JavaScript client and display to the user as well.
 - In general, the transitions have conditions, which are expressions. If you can give a homogeneous access to the objects around the transition, this can also be configurative, because there are many expression parser/evaluator solutions.
 - You should consider creating a homogeneous status reporting environment. Then, you can "ask" each related object (Person, Employee, Office, Shelf, Truck, whatever) about their current state; the Transition expression can access their data and state content and decide if the transition is allowed, and also report to the status container (what the user should do to enable this transition).
 - So, you can show the workflow to the user (because it is not hidden in magical configurations and source codes). You can inform them about the current state anytime without extra coding. And if there is no user, the batch processing can also use the same information: you replace the "user" with scripts, similar to the transition expression, but this time the result is the next state.
 - Still, you can use any "well known solution" for actually managing the workflow, so you get the solid, tested implementation; your extra effort is to convert your configuration to its native format. However, you are independent from the chosen tool; you can upgrade or replace it at a later time because you can forecast the efforts and side effects.

... my 2 cents...

Call/relay/broadcast within process or at the end

A few days have gone and still no Eclipse project in the new environment. Did I underestimate the power of the human factor (hard for me to start)? Or...

What I try to do now is to create the kernel features. The absolute minimum that my system must provide to an external developer for using just anything in the actual, dust-based environment. I have hit a serious question about control. When communicating with other components, I need to support the following tasks:

• direct call to another component, then continue the actual operation using the response (like evaluating an expression or alike);
• broadcast to a set of components (either through an iterated member field, or a result of a query), then either wait for the results, or continue operation, either synchronous or asynchronous way;
• set one or more relay components, either parallel or serial way (like in the interpreter, junction or sequence), either waiting for the return (and continue, like processing the closing block in a stream) or just let it go.

This can be done, my problem is with the "wait or not" question. If I wait, I need kernel functions to do this. If it is at the end, I need a special return constant to inform the kernel that it has to do so. I hate parallel stuff, and not straightforward solutions. If I support calls only, the kernel does not know if it has to do anything after the call, if it is at the end of the caller function. If I support return type only, the code becomes ugly and fragmented. None of them are acceptable.

Do I look in the wrong direction? A weird, but possible answer is: yes.

I keep thinking about providing a programming language interface, API for using dust, with generated codes, etc. At the same time, I think about an interpreter that allows fully functional dust programming above the language layer. What if I turn it upside down? 

I can say that dust has no programming interface. I have to "implement" my services using the declarative tools, without using any generated code. The only codes appear at the edges: when a component brings a functionality from the external environment (runtime, like JRE, threading, stream, etc.; or external toolkit, like a database connector, GUI library, etc.) So, I only need

• structure constants that the real codes can use to access data stored in their own connector data objects;
• connector constructs that help the loader to connect the functions/objects to the aspect message processors;
• and perhaps some generic tool functions that I always implement in all language environments to speed up development (in Java, StringBuilder manipulations, arrays, ...).

However, there is no need to support programmatic connection between custom codes. This is fundamentally modifies the role and core features of the shared kernel codes. That is:

• no entity evocation, 
• no message sending, call management, relay, etc - they go out to the core feature set available from the algorithm configurations...

This is most strange because I generally hate when configuration hacks replace source codes. On the other hand, I expose the structure to the framework and that looks great - I can do the same with the algorithms as well, that may be beneficial. My experience, and the currently growing number of interpreted script languages show that the common runtime performance allows this approach (the algorithm tree is exactly the same as a parsed script code). And there is still the option to generate source code or compiled binary from this tree... It is so unfamiliar because almost all codes that I have written so far or imagined in dust just disappeared.

The new approach also modifies the kernel implementation approach. So far I thought of the dust kernel as just another collection of components that talk with each other. Now it seems that the kernel components can be more interconnected as a Java implementation of this runtime, because there is no such thing as "dust code".

The implementation order:

• the shared parts (public include, shared Java toolkit project);
• kernel (with its modules perhaps in separate projects, but they can be interconnected on Java layer)
• external tools for the kernel (all for which normal dust communication is enough, like stream, log, etc)
• behavior (control execution, and expression evaluation)

Fundamental projects

Separate the Dust API from the Kernel. The first contains the communication layer and base classes, essential to communicate between components. The Kernel is the implementation of the API, it contains the main function and the initialization process. Important: it also must contain layers, because there can be various startup environments (for Java: desktop, servlet and GWT client are currently considered).

Start liking GitHub (server and client). Nice "for dummies" environment, perfect for me as I want to waste minimal amount of time on the environment, but make it safe. I use a new repository and also work with the wiki, both are better than enough for me.

On the other hand, it is hard to set up the toolchain. It's great having a mindmap on my tablet, but that is connected to Dropbox, not to GitHub. I type things here that I should add to wiki. In some cases I already feel the limitations of the wiki, I should use UML instead, but where - on the tablet or desktop? Better here because the UML plans should go directly to GitHub...

Dust restart

Besides logging my rantings on various forums, I decided to use this blog as it should be: a public web log. More precisely: keeping notes on my progress, so there is a place to go back when I go to a dead end.

So, I am restarting Dust (after countless previous experiments). The new start is motivated by the current achievements (and the further directions that are not reasonable to chase) at Continental. I managed to create a programming framework, that is seriously modular, mostly configuration-based (including the application structure like Spring and the GUI like XAML), mostly message oriented. A really big surprise: the JRE is very fast, although the fundamental comm layer is terrible (HashMap<String, Variant> value access, reflection-based function call), and that is still "good enough", even when this Java code runs in the browser (GWT client application).

So far so good, but the next step is upgrading the core with generated accessor codes, while that is an extremely complex stuff, and the framework is "good enough". I would not recommend this way to my bosses, though I would like to do it. This means coming home, and relaunch dust development, with the current experiences, but broader (aka: insane) scope.

Meta layer brought up another question.
I managed to separate the app structure from the language, not only the executor component hierarchy, but also the predefined messages, so sending them can also appear in configuration. I also integrated a script engine to the framework, so the "algorithm" became configuration, and (due to unpleasant reasons) I implemented a quick and dirty Excel formula parser and executor.
I am also sure that all source code is a serialized form of an algorithm, and the algorithm consists of only a very few tasks. This is why the source code can be compiled to machine code, and this is why a so simple language as C can be generally used to solve any task. (Not to mix the features of the programming language with the external tools and the runtime).

So, I want to implement the algorithm layer as well, and generate the language source from it. Finally, I want to use plain C, because that focuses on the control structure; it is harder to manage Java/C# sources, where the language and the runtime is merged.

Serialization brings grammars into view. I chose ABNF for it looks simpler than EBNF to me. Checking the elements that are required to build a parser, I found a close match with the control structures (block, sequence, repetition, junction). I want only one, and that should be the control components; grammar should be a user of control components.

I also found that relay comes into view. When parsing a stream, a junction (or the end of a repetition) means the rest of the stream can continue in multiple paths. My parser is already a SAX analyzer, the stream has the control. This means the junction root does not "call" the optional processors, no "trackback", but it should register multiple relays, and should get the control back when they finish (and in normal case, only one succeeds). In this way, the relay path represents the call stack (easier to analyze).

I have problems with separating the serialization layers: the grammar (control structures that can do stream i/o), the grammar of grammars (which is able to generate control structures from a grammar definition), reading arbitrary content (by a simple language), serialization (the same data language with smart content processors and some semantic rules), and code (control structures in a real programming language).

I can't see the separation, so I have to implement a working code to see the process. Today I am a Java programmer, so I will create this part in Java for the first time. I need a generic Dust runtime, and process ABNF, JSON and C. The required components (separate projects):

• API: base classes, fundamental services; standards for pseudo-generated codes
• Data: cloud, Entity/Aspect separation, aspect access, data management
• Process: call, relay (later intercept)
• Util: some of the generic util stuff
• Stream: read and write files; processor types
• Behavior: the Control and Expression components

Nice list.

State of Technology: Two Obstacles Gating Tech’s Future

On LinkedIn

Totally agree with the questions, but I'd like to offer some other answers.

As a programmer, I think the greatest problems are behind the smallest character, the dot: we write source code to build component structure, access data and call functions with it. So we bury our knowledge into an environment instead of sharing it.

As a kind of philosopher, I see that we have real problems, like climate change, eroding knowledge, motivation and culture or Fukushima - but we want to solve an artificial question: how to make money? As technical expert I had to realize that the second question is actually the cause of the previous ones...

So I think we should not search for new technology or infrastructure, we have enough of that. We need a new vision. Here is mine.

Mese az evolúcióról (1)

Egy történet, amelyhez szerintem a legtöbb feltétel adott ma is: Gron :-)

Sic transit gloria mundi

LinkedIn
Software Architects and Enterprise Architects Group
How would you describe a well-architected system?

@Ethar - We were asked to create a system for public touch screen terminals. The content should be managed on Java servers, the players to run Windows 8 C#. My pilot project was how can I create a remote configurable application in an environment I have never played with.

You should keep in mind that I have worked in C# for 2 months. So the code is ugly for anyone, and of course, just a POC experiment. The project came from a company that abandoned the whole idea, so I was allowed to put it to my playground. It was a long time ago, so please don't ask me about details, I am buried in Java now.

After all this, have fun: https://github.com/MondoAurora/DustFramework/tree/master/c%23

---

@Ethar - Sorry, the problem here is that I work for a company on internal development. I think it is not loyal, and almost sure illegal also to disclose details of it on a public forum. I am very cautious here to talk about things that I have already published, or very generic, and not coming from the actual development.

So, the latest public stuff is that C# project. The "core concept" was that the application structure should not appear as source code, but a configuration that an administrator can edit (and in fact, an automatism can build from various segments) on a Java server. The "player" should load this configuration and "play" it. To achieve this, I had to wrap the required components (scheduler, message dispatcher, layout manager, video, HTML, etc. player), and open the structure for custom component integration (sample is the clock); make the configuration serializable, and do a business logic-component late binding. This is what you see there.

Another idea. In private, I think I can send you a sample configuration file of my project, there is nothing special in it. Then we can continue officially: I always beg for an architect overview, but I am too small for that. But if you are interested... ;-)

---

@Ethar - Sounds reasonable, but I have a problem with it. I admit that I am talkative, but I don't like repetition, and all that I would write you can read here: http://dust-framework.org/ (this is my "dream project", the texts should do as the "narrative"). If you want to see codes, you can check it there also (the link points to GitHub).

Of course what I do now is far over what you can see there in code, and I should spend even more time on it because I have a very tight schedule. I don't see the ROI in the proposed "pair programming" - I do it in my work hours on my real assignment. Sorry about this, I wish we started this talk earlier.

---

@Edward - I think this is the best moment for me to say thank you for the conversation.

I totally agree with your statements, and I see that the difference between us is the definition of "return" which is naturally translated to sells well in business, and serves well in my world, to which "academic" is a good term. Even if I mean something else on it than universities today. RIP Randy Pausch.

Productivity and AI before dotcom crash? Perhaps you knew Cygron/Mindmaker, where I was the lead programmer on decision tree / transaction analysis / model executor tasks!

Good old times... data mining on a 300MHz CPU with 64MB RAM; first tricks of true parallel execution; interactive 3D charts before hardware accelerators and OpenGL, with a local engine... YES! :-)

---

You are right, I have occupied too much of screen space here, and although I still consider my stuff focused and practical, this does not meet your opinion. That's life.

Of course I could add some notes to the "dreamer" section, like that the mentioned data mining tool won in 1999 and was finalist in 2000 at Comdex in Las Vegas - perhaps you can guess there was a bit of sweat and real work behind... Or that in fact I *was* lead architect and programmer when we created the national agriculture data management system, and for example when someone asked "why didn't you use Spring for IoC container?" I had to answer: "when I designed and wrote it, there was no Spring, sorry. But if necessary, we can migrate, because it is isolated."

@Ethar, the invitation stands. If you are interested in the real stuff, I can send you a configuration file with explanation, and if you like, we can search for an official way. I really don't want to bring more disturbance here. Thank you for the comments, it was interesting and useful to me. Sorry if I was too offensive or single-minded.