So, I am restarting Dust (after countless previous experiments). The new start is motivated by the current achievements (and the further directions that are not reasonable to chase) at Continental. I managed to create a programming framework, that is seriously modular, mostly configuration-based (including the application structure like Spring and the GUI like XAML), mostly message oriented. A really big surprise: the JRE is very fast, although the fundamental comm layer is terrible (HashMap<String, Variant> value access, reflection-based function call), and that is still "good enough", even when this Java code runs in the browser (GWT client application).
So far so good, but the next step is upgrading the core with generated accessor codes, while that is an extremely complex stuff, and the framework is "good enough". I would not recommend this way to my bosses, though I would like to do it. This means coming home, and relaunch dust development, with the current experiences, but broader (aka: insane) scope.
Meta layer brought up another question.
I managed to separate the app structure from the language, not only the executor component hierarchy, but also the predefined messages, so sending them can also appear in configuration. I also integrated a script engine to the framework, so the "algorithm" became configuration, and (due to unpleasant reasons) I implemented a quick and dirty Excel formula parser and executor.
I am also sure that all source code is a serialized form of an algorithm, and the algorithm consists of only a very few tasks. This is why the source code can be compiled to machine code, and this is why a so simple language as C can be generally used to solve any task. (Not to mix the features of the programming language with the external tools and the runtime).
So, I want to implement the algorithm layer as well, and generate the language source from it. Finally, I want to use plain C, because that focuses on the control structure; it is harder to manage Java/C# sources, where the language and the runtime is merged.
Serialization brings grammars into view. I chose ABNF for it looks simpler than EBNF to me. Checking the elements that are required to build a parser, I found a close match with the control structures (block, sequence, repetition, junction). I want only one, and that should be the control components; grammar should be a user of control components.
I also found that relay comes into view. When parsing a stream, a junction (or the end of a repetition) means the rest of the stream can continue in multiple paths. My parser is already a SAX analyzer, the stream has the control. This means the junction root does not "call" the optional processors, no "trackback", but it should register multiple relays, and should get the control back when they finish (and in normal case, only one succeeds). In this way, the relay path represents the call stack (easier to analyze).
I have problems with separating the serialization layers: the grammar (control structures that can do stream i/o), the grammar of grammars (which is able to generate control structures from a grammar definition), reading arbitrary content (by a simple language), serialization (the same data language with smart content processors and some semantic rules), and code (control structures in a real programming language).
I can't see the separation, so I have to implement a working code to see the process. Today I am a Java programmer, so I will create this part in Java for the first time. I need a generic Dust runtime, and process ABNF, JSON and C. The required components (separate projects):
- API: base classes, fundamental services; standards for pseudo-generated codes
- Data: cloud, Entity/Aspect separation, aspect access, data management
- Process: call, relay (later intercept)
- Util: some of the generic util stuff
- Stream: read and write files; processor types
- Behavior: the Control and Expression components
Nice list.
