Confusions in the concepts of model execution and code generation.

There is quite a lot of  fuzziness and confusion in the general discussion around the concepts of model execution and code generation.

I try to straighten up these concepts and their relationships.

1)      Isomorphic mappings
First every programming language (with a working compiler) is a isomorphic mapping between syntactically correct piece of code and the corresponding compilation ( executable code). This means that for every compiler a reverse complier can be written and that compiler will produce a corresponding source code from an executable piece of code.

2)      Current (deterministic) computers cannot execute any kind fuzzy code.  This implies that the executing code has to be always at the lowest granularity and the whole execution flow is completely deterministic.

3)      Models by definition are abstractions. They drop detail to gain clarity in more important aspects of things. This way the initial models to serve their purpose must exclude details that the applications using the model finally must have. This implies that there cannot be any meaningful execution of a model as such.

I think that it is important to separate domain (business) model from the whole application. Of course it is technically possible to model the whole application in one model, but the increase of complexity is enormous. I don’t think that one needs any “application model” because applications are always very simple thing and they should be designed not modeled.

All this implies that model execution cannot be anything very productive. An totally other question is “graphical programming language”. This has been tried several times in the past and with very poor results. Here the amount of detail is in the key role again. It seems evident that we are able to handle very small amount of information in graphical form. The breakeven point is actually very low and after that the situation very quickly gets really bad. One very clear and also good attempt in this direction was IBM’s Visual Works for Java. It focused in graphical GUI programming. In demos it looked fabulous but everyone that really tried to do something real with it was quickly convinced that it will never work. Even a modest window with some functionality became a total mess.

This means that graphical presentations (at least for me) are vitally important thing. Class diagram in UML is quite sufficient. Collaboration diagrams theoretically are also good, but the current tools with these are very poor indeed. There should be powerful tools to produce collaboration diagrams and they should include a default option to present them as animations. Some time Rose had that but sadly that time passed long time ago.

Then the true key question is how to make the transition from graphics to textual programming sooth or almost invisible.

To concretizes all this let’s go through an example. Let’s start with a small model. Here is an abstract core model for a web- book store.

Here we have four classes: Person, Book, Walk and Purchase.  Objects in Person-class are the only entity type objects. Walk and Purchase classes are event type and Book classes is description type. Walk event is actually the person’s visit in the shop and the name of the class could also be visit. The association between walk and book represents the book that the walking person has picked up and as purchase candidates. When the walk ends it will create purchase event and transfers those books that the person finally is buying.  Here is the corresponding collaboration diagram:

The simple generation from class diagram includes following:

@Entity
public class Person {
private String name;
private Date birthday;
private String creaditCard;
private List walks = new ArrayList();
private Long id;

@Column
public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

and

@OneToMany
public List getWalks() {
return walks;
}

public boolean addWalk(Walk walk) {
if (!this.walks.contains(walk)) {
this.walks.add(walk);
if (walk.getPerson() != this) {
walk.setPerson(this);
}
return true;
}
return false;
}
public boolean removeWalk(Walk walk) {
if (this.walks.remove(walk)) {
walk.setPerson(null);
return true;
}
return false;
}
public Walk createWalk() {
Walk walk = new Walk();
addWalk(walk);
return walk;
}

In a similar way the code for the Walk class can be generated. There for instance can be generated the pickUp() method as follows:

public boolean pickUp(Book book) { }

This signature can be generated but there is now way to conclude the method behavior even if for a human through the semantics of the method name this think could be quite evident:

public boolean pickUp(Book book) {

this.myHand.add(book);

book.pickedUp();

return true;

}

Finally there is now way to generate anything for the application. The application has to have at least one GUI window that enables management of persons and books and pane for a walk and purchases. To get a decent result the GUI and its functionality must be design and implemented by humans.

Advertisements

Creating an extra high productivity & extra high quality SW development team

Background

We have actually known for a long time much of the concepts and theory of good software development. The journey of 50 year of computing has gradually led us into the point where we understand.

The result of this is model driven process that ends up to a 3-tier application implementation. The layers are: Application layer, Domain layer and Persistence layer. The key element in this approach is the unidirectional association between the application and the domain layer. Here the application layer is strongly associated with domain layer but the domain layer has no references out of it at all! The development process should start with abstract domain modeling. This phase won’t take more than 10 working days in average of a team to build. The development starts from just implementing this model. Then it continues incrementally with design and implementation of the application GUI and workflows. As the work proceeds the details are also added to domain layer that started with skeleton model. The experience from several projects is that the number of classes increases very little but addition of attributes and behavior is of course substantial.

Great opportunity for a top high quality & high productivity team

Currently most of the development of operational software for companies is done based on very old and proven bad paradigms. In practice majority of development is done with a waterfall approach. This is bound from different directions like, wrong idea of the nature of the work, planning and working tradition, buying, selling and agreement traditions. This all lead to situation were a carefully selected team of skilful professionals would have an outstanding advantage in competing on the arena.

The structure and the working process have been proposed and even used many time in the past. I will assembly here ideas from Frederick Brooks: Mythical Man-Mont, Grady Booch: Object Solution, Ken Schwaber & Mike Beedle:  Agile Software Development with Scrum and Kent Beck: XP.

The good principles and best practices for such team could be added up as follows:

  • Small team with member each high professional skills
  • A Scrum-like working team yet somewhat different duties and responsibilities between team members. One could call it “surgical team” like Brooks did. In software the “surgeon” would be call chief programmer & architect.
  • There could be two teams. One for domain & total architecture and another for GUI & work flow control details. The interests of these teams are clearly more loosely connect than inside these teams.
  • The working process should start with clearly separated time boxed abstract domain modeling. Then the implementation process should start with generating the implementing code from this abstract model as starting point and guide lines for the incremental and iterative development work.
  • Professional skills tend to be badly underestimated. Somehow in IT-field the dominating assumption has been (as perhaps is still) that most of software development is routine work, which one can do with short introduction to work. The truth is complete opposite of this! The software implementation is difficult design work from modeling until the last line of coding. It is mostly problem solving and successful people have both good skill and enough routine work behind them to be good at it.The ideal small team could consist of following roles:
    • Chief domain developer. This fellow is the “surgeon”. He has the main responsibility of the overall success.
    • Domain co-developers (1 – 3). These as the assistants of the chief developer. The domain developers discuss and make the key design decision together. One of these is the deputy chief developer.
    • Application developer who designs and implements the work flow control
    • GUI designer
    • Case dependent additional roles
  • The team is really functional after a few ramp-up projects as each member learns to fit and function in this particularly team. To replays any team member requires a fully professional candidate and a learning period with assistance of the leaving member or the rest of the team.
  • One open question (that I cannot provide an answer now) is whether such teams should practice their performance  in advance outside any customer projects.

Really good customer is a must

All this things doesn’t work without the final link in the chain. This is the buyer, the one that the whole thing is done for and the one that should be the most interested in the result.

Participating in such a process require real interest and committing to the process. This means that the organization appoints the most knowledgeable key people in the project and at least some of those fulltime and that the management of the company is actively participating in the process.

In an agile development process very important decisions must be made all along the process. Actually the development of an operation management system is always at the same time a huge opportunity and a steep learning process for the whole organization.

Choosing the customer (at least at early state of working this way) is actually more crucial than creating the best possible team.

Paradigms and Object Paradigm in particularly with quote from Grady Booch

The concept of paradigm is a tricky one. To characterize this in some way we can describe paradigm as a outmost explanatory framework or mechanism of modeling a complex thing. (see: http://en.wikipedia.org/wiki/Paradigm).

A paradigm offers building blocks to create a model of complex part of reality. These building block are in a sense fundamental to all models ( you could call them explanations as well) that are base on that paradigm.

The paradigms for same aspect are mutually exclusive. Let’s take a concrete example. If you want to create a 3-dimensional model of houses, you can us as you building material for example: Lego bricks, matchboxes or carbon. With each of these materials you can achieve good results but the working techniques differ and of course there are differences in the end results too. Now these three materials – the building blocks – represents here the paradigms. The first (and I consider this most important) result of this consideration is that is impossible to mix paradigms. Either you work with Legos or matchboxes but there no way of mixing these materials. This is universally true with different paradigms explaining the same aspect of the same part of reality.

The article in Wiki also mentions both paradigm shift and paradigm paralysis. It seems that the used paradigms are changed from time to time to more accurate or explaining ones. For instance people hade from the days of antique Greece a very commonly adopted paradigm of solar system. People believed that it is geocentric.  When time passed and people learned more about the universe they realized that their understanding of Solar system was wrong and they had to do a paradigm shift to heliocentric. This shift proved to be a difficult one at least for part of people (at least around Catholic Church). This ended in a paradigm paralysis. The reasons behind this were various and so were the explanations and excuses.

Now let’s consider the object paradigm. The paradigm shift happened from procedural paradigm. The key difference between the paradigms here is in how they describe model’s behavior. The older paradigm has a simple way of slicing the behavior in time and end up with time flow of events as the object paradigm describes the behavior as collaboration of named objects and their separated responsibilities. These ways of describing behavior are mutually exclusive. If you explain the behavior as flow of events there is no point in doing that all over again with collaborative object and their distribution of work. If this would be done one of the models would be completely redundant and unnecessary or even confusing!

I think that most of business application development today are done with OO-languages and most of the design is done in procedural way. My question is how far we are in the paradigm shift and how bad is the paradigm paralysis with procedural approach? The fact is that application development with object paradigm is from 2 to 10 time as effective as with the previous one! In the next post I will elaborate on this and start it with a quote – part of epilogue from Grady  Booch’s Object Solutions book.

  • Software development is difficult
  • Object-oriented technology helps

Is object-oriented technology mature enough upon which to build indus­trial-strength systems? Absolutely. Does this technology scale? Indeed. Is it the sole technology worth considering? No way. Is there some better technology we should be using in the future? Possibly, but I am clueless as to what that might be.

It is dangerous to make predictions, especially in a discipline that changes so rapidly, but one thing I can say with confidence is that I have seen the future, and it is object-oriented.

This was written around 1995. At that time the OO boom was at it’s heights and we all though that transition from procedural would be graduate, smooth and strait forward. It proved to be more difficult and there were a lot of forces against it. There were (some still are) the mighty stakeholder of procedural legacy.  The new concept proved to bee more difficult for many people than we thought.  Anyhow everything here hold today! of course we have learned something in 15 years and we can do things more efficiently today but the bases is here, solid and holds! The software that I am working with can and should all be implemented with OO.

OO domain models for several industries

I am about to start to publish OO domain models of some industries. The first model that publish is within healthcare and could be named as Heal Centre. The model will cover a group of doctors have reception. The model dynamic starts from a patient need for a doctor’s reception and it end at diagnosis, a treatment plan and possible recopies for medical treatment.

When I publish this kind of model, then it will contain the following parts:

  1. class diagram of the domain
  2. one to some collaboration diagrams describing the most essential business logic.
  3. PowerPoint animation presentation for each collaboration diagram to easy the reading of those.
  4. The abstraction level of this model is high and for instance the attributes in the classes is only a small portion of the required when implementing of any application that uses this domain.
  5. During a possible implementation the abstract analysis model should incrementally expanded to include all the necessary attributes. In most cases very few new classes are needed in this phase.

In this post I am not yet publishing the Heat Care (or any other real) model. Here I only give the form of the publication and the reasoning behind it.

I will publish here a very small toy (or tutorial) model called a BookShop to give you an example of everything that I described above.

The business that we look at here is a small bookshop in a centre of a city. The following is deliberately highly abstract model to emphasize the difference between the reality and the model. In the shop the books on the selves actually represent a publication than a copy. So our models don’t have these copies as objects in the model at all.

This way one can think that the copies represent a catalog. This way it is rather easy to sift from a real shop to internet shop. The model has first event called walk. This is the even where a person enters the shop and stars to walk around. During the walk the person pick up interesting book. This set of books are the candidates for purchase.

I use Peter Coad’s color coding. This will make it a lot easier to read. The colors are: Entity-classes green; Role-classes yellow; Description-classes blue and Event-classes pink.

Here is a collaboration diagram on what I described above.

If you have a need for a model on a specific domain then send me a proposal and let’s see can I do it for you.

http://personal.inet.fi/cool/jukkatamminen/OOstaf/webKauppAnim.ppt

http://personal.inet.fi/cool/jukkatamminen/OOstaf/BookShopUML.uml

OO model driven application development; the state of art and for the foreseeable future

OO model driven application development; the state of art and for the foreseeable future

Purpose of a model

There is (and has been) a dream of creating “application from models”. The trouble has been that the meaning of this phrase has not been analyzed at all. There is a serious mistake in OMG’s approach. For some curious unknown reason the concept of model is not clearly understood. The serious misunderstanding is about role of a model and then the subsequent amount of detail in the model. The main purpose of a model (even quite theoretically taken) is to simplify the part of reality that the model models. This is done by reducing the reality’s amount of detail to a manageable amount. This is as it should be and a good thing. But because of this fact a reverse is not (even theoretically) possible. Once the detail is gone there cannot be any way to return it back just working from the model itself.  This means that there is no way in which generation from more abstract model could produce (or generate) richer and less abstract models or code. This means that first we work hard to get rid of detail and then we work hard to return the necessary detail for each application. Now someone could ask: “isn’t it then more effective to work with the detail rather than go back and forth?” The answer is definitively no! As the complexity of the target domain grows the benefits of doing this grows exponentially!

The process is the following:

  1. Create an abstract domain model. This model simplifies the realty. The level of abstraction ( read simplification) is a controlled process. One can say that there is almost a natural constant (like for instance Plank’s constant in physics) for the number of classes in best models and this optimal number is between 30 and 60 classes. This number has nearly nothing to do with the particularly domain in question. This number mostly reflects human capacity to manipulate complexity in one shot. This domain model have to be a simulation model that fully (but simplified) describes the target domain both essential structure as well as behavior.
  1. 2. Use ALWAYS the following 3-tier architecture

  1. 3. Implement the domain model almost one-to-one. Of course the number of attributes have to increase because in analysis state we deliberately take only the most important ones not all.

  1. Design the required GUI conversations for all the necessary workflows
  2. Use some good OO-database to implement the persistence. Actually the wording here is wrong because the term OO-database is contradicting concept: (seeJ

The role of code generation during this process

The MDA’s false idea is that you can “generate more than you actually have”. This thinking somehow bypasses or confuses the simplification that is the core of abstraction. When you abstract you definitely simplify. This means that more or less detail is ignored and thus the model doesn’t contain this. Then there is no way bringing that back without human effort that knows it and places it back again.

There is definitely a role for generation from abstract domain model. But a proper scope of it has to be understood. From the model’s point of view everything in the domain layer can and should be generated. Of course this will still lack majority of attributes and consequently the corresponding methods as well. We can generate the class structure, all the navigation throughout the network, all the identified method signatures and getter and setter for identified attributes. This is substantial but of course lacks a lot. The most efficient way to implement the method bodies is to write the for example in Java. The implementation of persistence layer is rabidly chancing and SQL databases are becoming obsolete. So in most cases the persistence base solution can be generated but it can require so tuning at the volume testing and when starting production. The situation is worst in application layer. The first and a crucial point here is that the designing of a good implementation for a workflow is a difficult task and it have to be done very with implementing various solution.  This task is so complex that to achieve a good solution the only proper way to do it is an agile iterative and incremental. Of course one big drawback here is the primitiveness of all the current tools and development environments.

About understanding and communicating an object model

A model by definition is a simplification of reality. The main reason for model is always a simplification of existing or a design of something to be. Sometimes as an extreme a model can be one-to-one, but calling that a “model” is actually somewhat misleading.

The reason to have a simplification of something in most cases is the fact that the total amount of detail would be unmanageable and not understandable and for this most general reason we use models.

As we model concrete 3D things like houses we can have the model in 3D also, but the size of the model is far smaller than the size or the real thing. The downsizing evidently will disregard or destroy some amount of detail and simplify the model compared with reality . Another way of modelling these things is a mapping done from reality (current or future) to some kind of language. In the case of building this language consists of geometrical figures, which maps to realty. For instance a floor plan consists of together joined rectangles and a set of other graphical signs like door and windows.

To understand meaning of a model implies that the person in question is able to “see” the corresponding reality trough the model. If we use a mapping from reality to not self-evident language (like in case of house plans or maps), then the prerequisite for understanding the model is the knowledge of the discreption language.

Understanding is not a two-value function (on ; off) but a complex multi value thing. Thus it is very risky to say that a person doesn’t or a group do don’t understand the model. Rather we should consider to what extent someone understands the model.

It is obvious that the more complex the model is the more difficult it is to understand all aspects of the model. Even the “understanding” concept is difficult. The complexity of a model depends on two independent sources: 1) the domain in concern intrinsic complexity and 2) the chosen abstraction level of the model.

The complexity of the domain cannot directly be influenced. Indirectly this can be influenced by limiting size of reality to be modelled. The level of abstraction is completely in hands of the modellers. The semantic descriptiveness of a class and its level of abstraction has the following dependency with ease other:

This means that when chosen the level of abstraction of a class this with determine the semantic content or richness of the concept. So the lower the level of abstraction is the more semantic meaning the class name (&definition) will have. Of course it is good to have as much semantics as possible but the contradicting demand is to reduce the amount of less important details to promote understanding.

To balance the model between these demands the most effective choice is somewhere in between the extremes (the orange line in the figure). A very important aspect of OO-model creation is that towards the end of modelling a given domain the emphasis is on simplifying the model as much as possible without losing its semantic weight or value. This means that we approach a point where leaving something away will have a dramatic effect on the model collapsing it totally. Here the model has reached its complexity minim (at a given semantic level). If someone then demands to reduce the complexity of the model being too complex to understand this will mean no model at all!

The abstract object model consists of a class model and a set of collaboration diagrams. Class model contain all the classes ( concepts) found during the domain analysis and all the relationships between the classes. This class model can be represented in one or several class diagrams. Even if the model is represented in more than one class diagram the model itself is always one and connected. It is moderately ease to show that this model if it is correctly done is in the complexity minim of all models.

I will continue with understanding the model. First we have several different methods to model reality. Some of them are structurally simpler than OO-models like for instance use cases which have hardly any structure at all. Then we can move the whole model (ie. all classes) up and down the semantic-abstraction triangle. If we chose very high abstraction level we loosed a lot of detail and we will have a simple model. So simple models are far easier to understand than the models that have more structure, semantics and more fine grain behaviour defined.

The best OO-model in the middle of the triangle is most effective model. Thus it leads to most effective implementation as an application.

The model include also a set of collaboration diagrams describing a few cases with the deepest and most important sequences, which describes some instances of  creation of the most important event object of the business cycle. These diagrams consists of objects not classes. For this reason the are very concrete and far easier to understand. Sometime we can draw additional static object diagrams to explain complex structure without any messaging.

Associations are first class citizens – or even more than that

In our OO tradition, which is reflected and somewhat in UML-diagram, we neglect the value of the association. As we draw like this:

Image1

It leads us to think that the association is actually a handle or a reference to another object. In a simple case this is to large extent true. Actually a closer investigation into the association reveals that association can have attributes. Theoretically we can say that the meta-model behind our object model – the vocabulary of concepts – should include the following structure. Here association is a class-like structure but includes additional elements; the connecting arms the association ends.

Image3

UML includes also an associative class notation:

Image2

The first question raised by this is “Does this class represent the whole association”?  My answer here is yes. In this way we are able to give the association attributes and behavior. This association class is restricted to bilateral association. This means that the association is exactly between two objects. We can however think of multilateral association as well. I will return to this issue after introducing another concept.

Let’s take another way of looking at class diagram. Peter Coad proposed using color coding in class boxes (as I am using here). Coad’s meta-model for coloring is following: Here all classes are divided into four categories: entity, description, role and event-moment.

Image4

This categorization has proved to be very good and exceptionally practical. Of course four categories is very course grain and thus from time to time one comes across a situation where it is difficult to decide the category.

Let’s combine these viewpoints and analyze this structure a bit deeper. I think that both role and event categories are actually extended or specialized association types. Let’s take them one by one.

Event as association

An event object can be considered strictly speaking self-contained, but conceptually almost every event needs at least one participant to justify itself. So in practice event object are also kind of associative objects.  Here we meet a multilateral associative object. Event is an association in space-time continuum, that associates all objects that are connected by time and space.

Here is a modification of previous Person – Dog model.  I have added here an event that actually and conceptually is “behind” the association. To live together is the real association.

Image5

Emphasizing the time aspect here gives conceptual clarity and adds the duration aspect for free. Sometimes (perhaps in this case too) the time aspect increases anxiety in people but it definitely clarifies the situation.

Here we have another example. He person is going to have a meal. The meal is prepared by the recipe of that meal. He is the first variation of the class diagram. During the collaboration as it start person object needs a handle to the particularly meal to be able to sent it a message cook().

So here the association between Recipe and Person is purely to enable the message passing. Even if the earlier OO authorities suggested to have an association here I disagree. I think that association in  class diagram should reflect stabile and lasting structures in reality ie. they should have a meaning. In this example  there is no real stabile association between person and recipe. The association is momentary and temporary.

Image6

This is way I choose the next diagram instead. On collaboration diagrams also temporary association are seen as message pathways when needed.

Image7

Finally we can enhance this by adding an explicit event object class Cook here. Now the final comment here is then in this case the association between Meal and Person. This is more question of taste that right or wrong. I prefer rich association and would have myFood association even if this can be deducted indirectly trough objects from Cook – class.

Image8

Role as association

Role- class objects are not self-contained but dependent they will always need the actor association (the one who’s role is in question) and the action part where the role “is used”. Thus we can deduct that all role objects are actually special cases of an association object. The nature or role semantically is “participation in action” but it is quite common to shortcut this and put the role between to entity classes. The following is a typical example of this:

Image9

As noted actually this should be as follows:

Image10

Now the role type object is in between entity and event. In this special case as the cardinality of positions in employment is exactly one so the role object is in this case completely redundant. All its attributes can be moved to event itself.

Image11

On contrary when the even has more than one participant the role object has a true meaning and cannot be by bypassed. Technically the role attributes could be move to person, but conceptually it is more accurate to separate to attributes from person to role because one person could have several roles active simultaneously and in time sequence as well. In this way role object actually is a degenerated event within the even.