Feeds:
Posts
Comments

Archive for the ‘Software architecture’ Category

I have been using Netbeans IDE for many years and it is my IDE of choice. I can’t explain why Netbeans and not e.g. Eclipse or IntelliJ. It does not mean I consider other IDEs as inferior – not at all. For example I and my colleagues have chosen Eclipse RCP as a platform of one of my more significant projects – Sabre Red Workspace.  I’m just personally more comfortable with Netbeans and I see it getting better and better over years.

So recently I learned two good news:

    1. Oracle decided to continue evolving Netbeans (which was owned by Sun previously). This was not obvious since Oracle has its own IDE (Jdeveloper) and they might have decided to terminate Netbeans. I’m glad it is not the case (thanks Oracle!).
    2. Netbeans finally adopted OSGi as its runtime module container. This is good since it had its own solution for this and the team was evolving it over years while most others converged on OSGi. This looked suboptimal at least. Also using OSGi potentially brings possibility of using Eclipse plugins in Netbeans, too.

It seems after Oracle acquisition the future looks good for Netbeans.

Slides are here.

Read Full Post »

Comparison of software tools (languages, IDEs etc)  is one of the favorite topics on various forums. Sometimes it takes a form of a heated debate akin to those between Catholics and Protestants during religious wars of 15-16 centuries (e.g. see this post and comments to it). It is no surprise taking into account emotional attachment many developers have to their favorite tools.

I have never seen much sense in most of such discussions, though.

Consider this example: what tool is better: screwdriver or hammer?

Depends on what you want to do, right? OK, let’s suppose you need to hammer in a nail, so the hammer is more appropriate. Or, maybe a nail gun? The choice of the hummer or the nail gun will depend on which one makes sense from economics standpoint (if you need just to hammer in a single nail once a year, buying a hammer is better justified; if you need to hummer in 100K nails, a nail gun sounds as a better choice despite its higher price).

Also it depends on availability of people able to operate the tool: what is the use of nail gun if you could not find anybody able to operate it? Maybe a hummer could be a better choice in such case?

What tool is better: 1 pound hammer or 2 pound hammer? Depends on an individual which will be using it: in general case the 2 pound hammer is more efficient, but it may be too heavy for the given individual to handle; then 1 pound one is better.

This tells us that advantages/disadvantages of any tool must be evaluated only within a context which includes:

  • Task which we want to accomplish with the tool
  • Availability of people able to use the tool
  • Economics of the process
  • Capabilities of the individual or the team who are going to use the tool

There are different software development tasks, different skill levels of developers, different economics of the process, different skills available… That’s why comparing software tools (languages, IDEs etc) outside of a concrete context does not make sense to me.

Unfortunately this is exactly what happens in many discussions about software tools. People come from different contexts and they start comparing tools without defining the context explicitly. And it results in somebody absolutely convinced that the tool ABC is the best thing invented since sliced bread and her/his opponent absolutely convinced that both the tool and the former individual are evil and totally wrong :)

Read Full Post »

We have discussed  craft software development in several previous posts. While I personally like software craft production methods, as a software architect I want to understand how to make software development more efficient in mass production environment.

Software mass production has several distinctive characteristics:

  1. Developer working in such environment faces large amounts of code she/he is not familiar with; often it is legacy code written years or decades ago by somebody who is not available anymore. There is no much time to learn the code; the developer must become productive on that code very soon.
  2. One can’t assume that a developer working on the code will have skills level higher than average; having a developer with low skills must not be a risk.
  3. Cost/time to market considerations are of paramount importance.

The first point means that developers often are not familiar with the code they are supposed to work with and have very little time to learn it. This is a big problem, but there are ways to alleviate it.

The code should be broken into relatively small modules with very clearly and explicitly defined boundaries (even at the level of code repository) and contracts between them. My observation is that time necessary to learn unfamiliar code grows at least as square of the code size as the latter increases. Having it broken into well encapsulated modules helps to reduce the amount of code that has to be learned.

A better design also helps to understand the code (see here) as well as other basic techniques.

All necessary elements of the code must be explicitly described by the code. You think it is always the case, do you? Not at all. Think of dynamic languages -  for those languages type definitions that are important parts of the code (code metadata) do exist in developers heads only and never expressed/documented explicitly. Or think of implicit agreements that are used in Perl. You can find much more examples if you look around.  While such things may be not a big problem when a developer knows the code she/he works with well, this is a big problem in software mass production and as such must be eliminated.

The second point means that the code base and the tools must be such that even a beginner developer will produce a code of good quality.

While having code modular and explicit definitely helps, I think this is more about proper tools. Think, for example, about differences in how strings are defined in C++ and Java. While C++ strings caused countless problems, a way strings are defined in Java just eliminated completely the type of issues.

Basically for software mass production we have to choose deliberately tools that do not allow common problems to happen or at least warn about possibility of them happening. We better help the developer to avoid problems than rely on her/him to fix them.

This is not just about software languages. Think, for example, about differences between Maven and Ant. Ant is a sort of free-form tool while Maven is much stricter and prescribes certain ways of doing things. Maven is obviously better suited to software mass production.

The third point is of paramount importance for software mass production while it is of little to no importance to open source and lesser importance for software craft production.

Let’s suppose your company has a problem it wants to solve by developing certain software. There are two options: a technically excellent solution (architecture/technology/design etc) and just satisfactory one. Which one would you choose? The first one? Not so fast – you haven’t yet considered all important inputs for the decisions. For software mass production we have to consider cost and time (this is business after all).

Let’s suppose the first solution will cost $10M and it will take 5 years to implement while the second one will take 9 months and will cost $0.9M. Now advantages of the first solution are not so obvious, right?

Let’s add one more input… Your company has just $1M budget for developing the solution and if it is not available within a year, your company will be out of business. Given this, which solution is better?

Cost/time considerations are important for all choices we make in software mass production, not just big ones. They are exact reasons why developers working in software mass production environments e.g. should use modern IDEs as opposed to just vim or another text editor, should use object-oriented languages as opposed to procedural ones (the procedural ones are 5 times more expensive to develop – see here), etc, etc.

Probably, there are more issues that should be considered if we want to have efficient software mass production. But those three look to me as the critical ones. Disregrad of any of them will likely cause many problems.

Read Full Post »

Recently I have read an interesting article: Peggy Noonan On Steve Jobs And Why Big Companies Die by Steve Denning which I highly recommend.

However while I agree with the main Steve Jobs and author’s  idea that quality of an underlying product is of paramount importance for long term well-being  of a business, I don’t think that this is the whole story. There are examples of demise or near-demise of great companies for reasons that have nothing to do with quality of their products.

Couple of examples. A recent announcement of AMR/American Airlines bankruptcy. Their products (flights) are on a par with the industry. What caused them to go into bankruptcy is an unsustainable financial model.

More personally sad example for me was a demise of Digital Equipment Corporation since I used their computers and loved them (I still have fond memories of them).  It might be all but forgotten name by now, but in 60s-70s-80s it was a major player in IT industry rivaling IBM. In a way it was a company very similar to Apple. DEC was created by two engineers - Ken Olsen and Harlan Anderson. Ken Olsen who was a great visionary and a great business leader similar to Steve Jobs, led the company through almost all its life. The company had great products (computers of PDP and VAX series), but sadly its leadership underestimated importance of emerging PC market…  All that remains of this great company after a series on M&A ended up as a part of HP.

Another story may be a near-demise of Ford company when another great engineer and great business leader Henry Ford  failed to recognize that Model T times were gone.

Therefore I disagree with both Steve (Jobs and Dunning). All three facets of business (product, finance, marketing/sales) are equally important. Neglecting any of them – that’s what kills great companies. Sounds much less sensational than the article’s conclusion, but agrees better with facts.

Read Full Post »

We have discussed craft production of software in this and this posts. As I mentioned, it has many advantages, but it has one big problem: it can’t scale beyond certain limits at both individual project level and at industry level.  There is simply no way IT industry can employ exclusively expert developers  capable of doing software craft production. It must use developers that can’t be software craftsmen.

This may look like an obvious point, but I found that many folks in IT industry have hard time agreeing with it. Let’s discuss this in more details.

Let’s assume your company is planning to develop a software product and you are thinking of how to staff the development team. In one of my previous posts I have mentioned that industry average is 15K lines of source code per member of development team per year. One can say that if we calculate throughput just per developer, it will be higher. True, but not completely. Other members of a development team perform certain functions that otherwise developers should have performed. If there is no business analysts and testers on a project, developers must perform business analysis and testing, right? So even if we compose the entire team just of developers, it is unlikely that an average throughput per team member will be much higher.

Another popular argument is that best developers have much higher throughput than average. This is also true. But how much higher? There are opinions (Tom Love) that the best developers are up to 25 times more productive. I’m a bit skeptical that such big difference exists in reality, but there is the difference indeed. I had a case where I could measure the difference with certain statistical reliability. I have been observing a team composed of 7 developers who did just bugfixing and nothing else for 2 or 3 months. One developer out of this group fixed 6 times more bugs in total during the time period than each of remaining 6 developers in average did.

Still, let’s be generous and assume that best developers can steadily outperform industry average by 10 times.

This means that if you plan to complete development of a product having 3 millions lines of code in two years, you will need 10 outstanding developers or 100 average ones. You bet you would have hard time hiring 10 outstanding developers. But what if your company is planning to develop 5 such products in parallel (which is not unusual for big modern companies)? Possibility of hiring simultaneously 50 outstanding developers is really very remote.

One may argue that nowadays most software development is about maintaining and evolving  existing software products. True. However there are limits to amount of code which an average developer can efficiently maintain. Per Tom Love it is up to 50K lines of source code per developer.  Again let’s assume that best developers are capable of maintaining 10 times more (500K). Well, how many developers would we need to maintain 20M lines of code? 40 exceptional ones or 400 regular developers.

Well, again somebody may argue that the solution is in better candidate selection process (e.g. one of Tom Love’s ideas as well).  It is easy to show that this does not work for bigger development teams.  And here is why.

Let’s assume that your selection process is very good and it selects a single expert developer with probability Px. Then probability of hiring a team consisting of n experts is p=Px**n (Px in power n), which is an exponential function. Here are probabilities of hiring all-expert teams of different size for different probabilities px  (0.9, 0.8, 0.7, 0.5):

As you can see, even a very good selection process with Px=0.9 gives us less than 50% probability of success while hiring a team of 10 expert developers. In reality I would say most of selection processes have Px around 0.5, so hiring a team of 3 expert developers would be problematic.

An interesting consequence of this is the fact that small startup companies have much higher chances to have all-expert development teams. This is the important fact which itself has  interesting consequences we’ll discuss in further posts.

Also, accordingly to Law of Large Numbers,  the more developers we hire, the more their skills will be trending to the industry average.  “Nothing personal, just statistics” :)

All the above says to me that it is possible to assemble a substantial team of expert developers for certain projects for a limited time; however it is unrealistic to expect that nowadays the whole IT industry will be able to staff  developer teams with expert developers or that a substantially large development team may be staffed by experts for a long (many years) period of time.

Conclusion: modern demand for software development can’t be fulfilled by software craftsmen alone.

This is not an unusual case. Many industries were using craft production initially (e.g. car manufacturing, aircraft manufacturing etc), but at some point that had to employ another organizational forms of production to be able to fulfill growing demand.

We will discuss this in later posts.

Read Full Post »

I have got a number of comments to my previous post from this series. They mostly revolved around definition of software craftsman. This is an interesting topic which probably deserves a more detailed discussion.

To start with, we have to distinguish between  software craftsman role and qualities necessary for an individual to fit into the role. The role is a developer producing a code in  the craft way. Not each developer is capable of doing this successfully. At the same time not each developer capable of software craftsmanship does actually act as craftsman; e.g. she/he may work in a company which does not produce software in the craft way. Only when the role and the capability come together we have a real software craftsman.

What is craftsman in a broad sense? It is an individual which is producing a relatively complex product (clock, knight’s armor, beer, glass vase, shoes etc) on his/her own from start to end with good quality as far as end user of the product is concerned.

If we translate the definition to software development, I would say that

Software craftsman is a developer capable of architecting, designing, implementing, testing and deploying to production a substantially large software product of good end-user quality on his own  (with possible aid of business analysts).

Probably all would agree that this requires an individual having considerable skills and expertise, i.e. an expert developer in a broad sense. I also think that most would agree that not each and every “rank and file” modern developer has such level of expertise. In fact, my observation is that just a small fraction (maybe not more than 10% or 20%) of overall software development populace can be classified as all-around experts.

The definition  above refers to quality of the product which in my opinion contributes further to the confusion of opinions  around what is software craftsman. This is because quality of software is a complex multi-dimensional notion and many people mean different things while talking about quality of software.

There are two major  aspects of quality of software: external quality which is quality of the software as seen by its end user (end user quality) and internal quality which is quality of the software as seen by IT folks, i.e. developers, testers, operational support engineers etc.

Understanding the difference is important since those two aspects of quality do not necessarily come together. A perfect example can be found on many web sites. Look e.g. at the web page were you are reading the post (the page design is courtesy of WordPress which I’m using to publish the blog; all credit for the design goes to WordPress).  The page looks good, its purpose is clear and it is very easy to use for an end user, right? Now take a look at the page source . What would you see? Rather not-so-clean HTML code with quite a bit of hardcoded CSS styles, JavaScript scriplets etc. External quality is good, internal quality is rather not as good.

Software craftsmen invariably strive to build products of good end user quality. Of course, this is not always possible given many external factors like corporate budgets, release schedules etc. But still, they build as good products as possible given the circumstances they operate within.

Internal quality is a different matter. Often, when a software craftsman works alone or with just few other craftsmen, they may produce a code which does not look so good from a standpoint of quality standards of mass produced software. There may be no comments, few or no unit tests, a lot of non-standard components, no automatic build, no automatic deployment etc.  However if the software is written with the intent of it being used by other IT folks (e.g. open source projects), internal quality of such software when produced by craftsmen is almost invariably good since in this case the craftsman considers other IT folks as end users.

By the way, software which looks good for a developer may have poor quality for other members of IT industry as well. Surprise? Well, this is what often happens in reality. Even well-written software may be a nightmare to automate its testing or to deploy/upgrade etc. For example, how many times you have seen software which is very easy to roll back and to make its previous version operational in no time in case problems were found after a new version was deployed? My experience is that this capability is very rare while it can make life of operations folks much easier.

There is much more than that to quality of software and I can spend considerable amount of time on discussing software quality, but let’s return to a definition of software craftsman.

Ability to come up with an idea for an innovative product is not a necessary part of software craftsman’s skill set. In fact, very few craftsmen in broad sense as well as software craftsmen have ever come with innovative product ideas. Their distinction is rather in building/manufacturing  products rather then inventing them.

Formal training in particular area (IT in our case) is not a mandatory pre-requisite as well. True, having good formal training definitely helps (e.g. both Bjarne Stroustrup, creator of C++ and James Gosling, creator of Java, have Ph.D. in Computer Science) . However I have known several very good developers who were essentially self-taught.  For example  true IT legends, creators of C and co-creators of Unix  Dennis Ritchie  and Brian Kernighan. The former was trained in Physics and Applied Mathematics,  the latter was trained in Engineering Physics and Electrical Engineering.

This is true for other occupation as well. Charles Darwin was trained as priest, Arthur Conan-Doyle was a doctor, Charles Chaplin didn’t get any systematic education whatsoever.

Before I finish the post, I’d like to mention an interesting observation I made. I have noticed an interesting feature which I almost invariably see in software craftsmen: they very rarely if ever use debuggers. As soon as I see a developer who successfully and quickly writes code without resorting to a debugger, this almost invariably means that the individual can potentially work as craftsman.

Why is this so? This is because software craftsmen deliberately attempt to write code with very few bugs and  to write the code in such way that the bugs could be found and fixed very easily without any debugger. They do this because it is much more time efficient to write bugless code than to write the code with bugs and the fix them applying complex and slow tools such as debuggers etc.

When I say that it is possible to write code with just very few bugs  to a broader audience they often look at me in incredulous manner and often challenge me on whether this is possible at all.

Yes, this is possible and I know this for sure. It is not too simple to learn because it requires making certain practices an automatic part of how a developer does her/his work, but still this is not a “rocket science” and can be achieved within a year or two. I have mentioned certain such practices in one of my previous posts; good code design helps as well.

If you are interested to find more about practices helping to write a code without bugs, I would highly recommend you reading the book by Glenford Myers. It is an old book by IT standards, it was published in 1976 (e.g. it was written before object oriented design, internet and PCs and even before some of modern software developers were born :) ) and sometimes a bit outdated, but the bulk of the recommendations is still very valid and rock solid right.

Read Full Post »

Software development exists for around 70 years. Its purpose is to produce ( to manufacture) software and from this standpoint it is just one of many manufacturing industries. Car industry makes cars, clothing industry makes clothes etc, software industry makes software.

Despite this software development is still positioned apart from manufacturing industries as something very different. Even the terminology is different. Software is not manufactured, it is written or developed, right? Agreed, there are specifics of software development that keep it apart from shoe manufacturing. However there are specifics in shoe manufacturing that keep it apart from computer manufacturing, right? In my opinion, there are many things in common and looking at parallels between software development and manufacturing may bring better understanding where software development is as an industry and where it might be heading.

Let’s for example, look at organizational forms of software development.

There are different organizational forms in manufacturing industries in general. Arguably the oldest one is craft production. That’s what Wikipedia says about it:

Craft production (or One-off Production) is the process of manufacturing by hand with or without the aid of tools. The term Craft production refers to a manufacturing technique applied in the hobbies of Handicraft but was also the common method of manufacture in the pre-industrialized world. For example, the production of pottery uses methods of craft production.

A side effect of the craft manufacturing process is that the final product is unique. While the product may be of extremely high quality, the uniqueness can be detrimental as seen in the case of early automobiles.

As many other manufacturing industries, software development started as a craft production of the software code. The code was often “manufactured” by hand with little aid or without aid of tools (IDEs, debuggers etc).

For example, in 1980s I worked with a colleague, an outstanding software developer Viktor Krivcov. He was writing code exclusively in assembler language using just a basic text editor. No IDEs, no debuggers, no builds etc. He had been developing code with astonishing speed. For example he developed ASPECT – quite complex (for that time) interactive multi-user CAD system for simulating electronic circuits – almost single-handedly just within a year. The system was written in an assembler language for BESM-6 mainframe. Interestingly, Viktor steadfastly refused using any high-level language available at a time on the mainframe (FORTRAN, ALGOL, C – to mention few) and I suspect he considered that coding in them is not for real software developers :)

Please note, while talking about craft production, I do not put any negative connotation to it. In fact, I am a co-signer of Manifest of software craftsmanship and I share its values.

Craft production of software is still well alive and flourishing. It often produces products of extremely high quality as Wikipedia stated about craft production in general. Just several typical examples: Nginx , Redis and RabbitMQ.

Nginx is a web server which is an approximate functional analogue of Apache web server but it performs the work and scales much more efficiently. I did certain scalability tests of both in the past and I was really impressed by Nginx. Nginx is one of few web servers designed to handle so-called C10K problem. And guess what: it was developed singlehandedly by Igor Sysoev (probably there are other developers working on it by now).

Redis is an extremely fast NoSQL DB. Its initial version has been developed by Salvatore Sanfilippo alone.

RabbitMQ is very fast, scalable and flexible messaging middleware written in Erlang. Its initial version (prototype) was developed by just two developers ( Matthias Radestock and Matthew Sackman) as far as I know.

What are distinctive characteristics of craft production of software?

  1. All developers are highly skilled, often with education level higher than average (e.g. Viktor Krivcov has had Ph.D., Matthias Radestock has had Ph.D., Matthew Sackman is in a process of getting it). Of course, being highly educated does not necessarily mean having a degree.
  1. All developers are highly motivated
  1. Very small teams used, often consisting of one or few developers
  1. There is no separation of roles within the team as a rule. Technical architect, quality assurance and often business analyst functions as well as many other functions are performed by the developers
  1. Choice of technologies and tools is often (but not always) is very special, sometimes outside of the current IT mainstream. Redis and Nginx are written in ANSI C (not even C++!), RabbitMQ is written in Erlang, ASPECT had been written in assembler
  1. The teams tend to create and use their own tools instead of existing ones.For example, Redis team developed their own load test tool for Redis despite many load testing tools available
  1. Very little hardware is used (and, consequently, less investment is required). Things like certification environments, staging environments, UAT environments are almost invariably absent

#1 and #2 are critical mandatory pre-requisites for craft production of software being successful. C to G are consequences of the pre-requisites. If A and B are present, #3 to #7 come along naturally.

Why, for example, developers employed in craft production of software often use technologies outside of the current IT mainstream? Two reasons:

  • They choose technologies that are the most suitable for the task and not technologies that are the most fashionable or the technologies they know better. RabbitMQ team had chosen Erlang because it produced incredibly fast networking code, Salvatore had chosen ANSI C since he wanted to eliminate all overheads added by more advanced languages like C++, Viktor had chosen assembler since only it allowed him to build the system he envisioned within limited memory and CPU speed of the mainframe designed in 1967.
  • Their advanced skills allow them to be comfortable and productive with technologies that are hard to use for developers with average skills

Craft production of software is very cost efficient and this is why it is often used by small startup companies. There are actually two main variations of them:

  • Startup companies that produce software for IT industry (like Nginx, Redis, RabbitMQ). They sometimes consist of just the development team and almost nobody else
  • Startup companies that produce software for other industries e.g. aviation industry. Such companies almost invariably consist of a subject matter expert/business analyst who is not a developer but an expert in the target industry and a development team

Craft production of software has many positive sides. But it has a big problem, too, as far as IT industry is concerned. It does not scale to the whole IT industry.

It is a minority of developers who satisfy A and B requirements above. My observation is that probably not more than 20% and likely just 10% of overall development community fit to the software craftsman role.

Certain companies do large scale software development and they naturally need many developers. Unfortunately it is impossible to staff big development teams with highly skilled and highly motivated developers only. There is just not enough them on the market. True, probably certain companies like Apple or Google could do this due to their standing within IT industry. But by and large it is impossible.

Even smaller companies that need just few software developers often could not hire developers capable to craftsman software development.

So if software development industry wanted to scale, it had to find ways how to produce decent software with just average skilled and average motivated developers.

This problem is not unique. At some point almost any industry faced similar problem. For example car manufacturing. It started with small craft shops. But soon with growing demand they found that there were not enough skilled workers able to produce cars on a large scale while employing craft production organization.

The answer many manufacturing industries found to such challenge is mass production. We’ll discuss mass production of software in the next post. See also the post about definition of software craftsman.

Read Full Post »

Disclaimer: the blog post represents just a personal opinion of the author. If you disagree with what you are going to read in the post below, this is fine: you may be right and I may be wrong; I have no desire to argue with you.

Being software architect, I have tried for some time to formulate my impression of Android OS as a technology. Unfortunately it is somewhat controversial.

Here is my opinion. Android OS is a strange case of a brilliant architecture at a strategic level with quickly deteriorating quality at deeper levels. The deeper you look, the lower quality of architecture and design you see. It becomes “curioser and curioser” as you go deeper and deeper.

Good software should be simple and elegant, right? Now take a look at Android application integration layer. It is anything but simple and elegant. Rather overly complicated and bloated.

Or look, for example, at design of Android SDK classes and APIs. Why, for example, such classes as Activity or Serviceare subclasses of Context class? It is like deriving Car class from Road class. Yes, those two entities work together, but deriving Activity and Service from Context is a violation of both SRP and LSP principles. Would not it be much more clean to pass an instance of Context into Activity as a parameter of onCreate() method? Similarly to how ServletContext is passed into Servlet as a parameter to init() method in JEE SDK – clean and simple design.

Often there is feeling that certain parts of Android architecture were designed by several people working simultaneously who never had chance to talk to each other. Look, for example at ContentProvider and Service. Both encapsulate certain data and functionality, both play a role of Model in MVVM pattern. So we have a substantial overlap in their purpose and functionality. But the only common parent they have is Object class…

ContentProvider class design is very indicative to quality of Android SDK design. It looks like somebody started with intent to provide an infrastructure for encapsulating data and accessing it through REST-type API. Seems like a good idea, right?

However somewhere in the process it was probably decided to use ContentProvider for encapsulating predominantly SQLite relational database. As a result Android SDK ended up with inventing a rather fancy URI-like way to query straight SQL DB. What for? Just to show that Android is different? The irony is that at the end of the day SQL is used under cover to query the DB…

Intent class plays multiple roles, so it would be natural to have separate subclasses of Intent class for e.g. message passing and publish/subscribe functionalities, right? But there is no such subclasses (look here for details).

There are more examples…

I wonder why having Google resources Android managers can not make it well architected and designed software?

Overall, Android internals do not look good. So if you are a developer who looks for a job to get fun and aesthetic pleasure from it, you may think twice before getting into Android development.

All this said, I’m still convinced that Android is going to be a dominant OS in mobile market. It is like early versions of Windows: terribly designed but unstoppable as a market force.

Read Full Post »

In earlier posts we discussed:

Today we are going to discuss various mechanisms through which parts of an Android application collaborate. For lack of better term we’ll call all those mechanisms together as “communication layer” (to my knowledge, Android documentation does not offer any term for this).

As we mentioned earlier, Android application framework implements several communication patterns within its communication layer:

Message passing

This is probably the first communication mechanism which a developer encounters while learning Android application development. It is used to start an Activity or a Service. It is implemented via three methods on Context class (which we discussed in one of earlier posts) and one method on Activity class:

In both cases Intent class plays a role of a message passed.

How does Android know which Activity or Service should receive the message? There are two ways how this is done:

  • Intent may specify a name of the target component explicitly. Such Intents are called “explicitIntents” and they specify a target component via setComponent() or setClass() methods of Intent class. This type of Intent-based communication mechanism does not implement late binding pattern while all others do implement it
  • Target component is defined by a rather fancy resolution mechanism based on Intent filters.

Please note, in case of message passing communication mechanism the message (an instance of Intent class) is delivered to only one component or not delivered at all.

Publish/subscribe

Publish/subscribe communication mechanism involves the same Intent class as a message and the same intent filters as a resolution mechanism defining target components, but it works quite differently. Intents used for publish/subscribe communication are called “broadcast Intents”. Those broadcast Intents can be delivered to BroadcastReceiversonly. Since BroadcastReceiver is a class and Activity and Service are classes as well, neither Activity nor Service can receive the broadcast Intent (Java prohibits multiple inheritance).

Also, unlike Intent used for message passing, a single broadcast Intent can be delivered to many target components.

In fact, Android implements 2 different publish/subscribe mechanisms each o which has two versions (regular one and sticky one):

You are still not confused, right?

Normal broadcast mechanism delivers the Intent to all eligible BrodcastReceivers asynchronously and they act on it independently, i.e. one BrodcastReceiver can’t affect either how another BrodcastReceiver is reacting on the Intent or whether the Intent is delivered to another BroadcastReceiver. It is a regular pub/sub pattern.

Ordered broadcast mechanism delivers the Intent to eligible BrodcastReceives in a sequential manner, i.e.toone at a time. Due to this each BrodcastReceiver can potentially:

  • Use results produced by previous BroadcastReceivers that worked on the same Intent
  • If needed, abort further processing of the Intent

By executing ordered broadcast, a component which initiated it gets opportunity to receive the result of processing the Intent by all BroadcastReceivers in the chain.

Ordered broadcast actually is an implementation of chain-of-responsibility design pattern.

Sticky broadcasts are versions of the corresponding regular broadcasts. The difference is the Intent is available until it is explicitly removed by removeStickyBroadcast() method of Context class. Please note, this type of broadcast should be used with great caution since not removing the sticky Intents will result in a memory leak.

Conclusion on Intent-based communication mechanisms

All communication mechanisms described so far make use of Intent class as a message. Please note, that majority of corresponding methods are implemented by Context class so they can be theoretically used by all subclasses on the Context class.

By this point one may start thinking that all Android application communication mechanisms are using Intent class. Not at all.

Late binding of ContentProvider

ContentProvider class is essentially a wrapper around SQLite database in most cases although theoretically it can use other persistence mechanisms to store/retrieve data. You can read more about the class and its usage here.

There are many ways how Android code can use ContentProviders. Some of them include getting an object which acts as a proxy to the ContentProvider (normally it is an instance of a subclass of ContentProviderClient, which implements proxy design pattern). Some of them return an instance of Cursor class which allows iterating through data set returned by ContentProvider. In all cases the concrete ContentProvider which is going to be used is identified through URI (see details here).

Overall, it is a typical late binding pattern.

Late binding and Inter-process Procedure Communication (IPC) of Service

If you think it’s all communication mechanisms, you are mistaken.

Apart of ContentProvider class there is Service class which architecturally plays the same role of Model in MVVM architecture pattern. We have discussed the hypothetical reasons why Android has two different classes to fulfill the same role in the previous post here.

There are two types of Service and each requires a separate communication mechanism:

  • Local Service which runs within the same process where it is invoked. Architecturally it is similar to ContentProvider apart that it has its own lifecycle and it is not necessarily backed by DB. A version of late binidng is used to access the local services
  • Remote Service which runs within another process from where it is invoked. Such type of service is accessed through a sort of IPC call implemented using AIDL (Android Interface Definition Language)

It is worth mentioning that the same Service may be used as both Local and Remote; the difference is in where the component which invokes the Service is located (within the same process or in another process).

You can read more about using Services here and here.

Read Full Post »

OK, we looked at Android application architecture background and architectural patterns implemented by Android application architecture. Now let’s talk about main parts of an Android application.

In this post we’ll introduce main “personages” of Android application architecture.

Android application usually consists of:

Java classes

The diagram below represents a class hierarchy of main classes from Android SDK a developer has to deal with while developing Android application:

There are much more classes than that, but those are the main ones. Classes highlighted in yellow are those a developer has to deal with (e.g. extend) directly. Other classes are important as well, but they are relatively rarely used directly.

View class is a base class for all GUI widgets. Android application GUI is a tree of instances of its subclasses i.e. GUI widgets. One can build the tree programmatically, but this is not how you should do this. GUI is normally defined by an XML GUI definition (layout file) and inflated (Android term) automatically into a tree of corresponding objects at runtime.

Activity class and its subclasses are the ones that provide logic behind the GUI. Actually it corresponds to ViewModel in Model-View-ViewModel architecture pattern (MVVM). Relationship between subclasses of Activity and GUI layout is 1-to-1; normally each subclass of Activity has one GUI layout associated with it and vice-versa. Activity has a lifecycle.

During the lifecycle Activity can be in one of three states:

  • Active and running when its GUI is in the foreground (technically speaking – on top of activity stack)
  • Paused if its GUI lost focus but it is still visible. No code is executed in this state
  • Stopped if its GUI is not visible. No code is executed in this state

Activity code runs only when the Activity GUI is visible and has focus. Also there is no guarantee that Activity object and its associated objects are in memory while the Activity is paused or stopped (very important point to remember; we discussed this Android memory management peculiar feature earlier).

ContentProvider class and its subclasses correspond to Model in MVVM architecture. In most practical cases it is a wrapper around SQLite database with rather fancy URI-based way to query the database. Theoretically nobody prevents a developer from building ContentProvider which will be using something else instead of DB for storing data. However given that query() method of ContentProvider returns Cursor object which is quite similar to JDBC ResultSet interface and the way how the query is made, nobody would doubt that the real purpose of ContentProviders is to encapsulate a database.

I don’t know how Android team came to such design, but my guess is here we have a combination of two good but not very compatible ideas.

Here is why I think so. The whole idea of content provider is likely inspired by AJAX application architecture. AJAX applications normally use MVVM architecture where Model is represented by particular URIs on a server side (however this has changed somewhat since introduction of HTML5 which allows storing data locally). Indeed, the fact that ContentProvider class is queried by URIs and makes extensive use of MIME types points to AJAX as an origin of the design. Remember, Google folks were developing a lot of AJAX software, e.g. Gmail, Google Docs etc, so it was natural to borrow ideas from AJAX application architecture for them.

Probably somebody else came with another cool idea: how great it would be to have a fully functional relational database running on a mobile device! (note, this was happening at around 2005 when mobile phones were much less powerful than now). And as a result they combined two good ideas into one ContentProvider class. As it often happens in software development combining two great ideas not always results in a good design; in Android case we have rather confusing ContentProvider design.

Service class and its subclasses are something which I have difficulty to classify. I think, Google folks have the same difficulty (please read their javadocs). They classify it by mostly describing what it is not. I would personally classify it as a special type of Model, serving a bit different use case than ContentProvider does.

In my opinion, architecturally design of Android Service was inspired by OSGI services.

I think Service is the solution Google folks came up with to solve a logical problem they created in Android application architecture with its threading model.

Think of this: Activity is active and running only when its GUI is on foreground. As soon as another Activity GUI comes in front of the current one, the current one stops running even if it was doing something. And what if you need to perform certain operation even when the process which tries to perform it is not on foreground? You can’t do this with Activity. You can’t do this with ContentProvider as well since it does not have its own lifecycle and it can run only while Activity which uses it is active.

Here Service comes to the rescue. It can be executed even when the process it runs within is not on a foreground. So if you develop Activity which must perform a long running operation which should be completed even while running in background, you should create Service implementing that operation and invoke it from Activity.

Service has a lifecycle as well. This means that it can be instantiated and run by Android application framework under certain conditions (we’ll discuss this later).

As I mentioned, as a Model, Service is somewhat of more general purpose than ContentProvider. It can use DB, but its API is not bound to DB as ContentProvider API does. In many cases Services are used to connect to external servers.

BroadcastReceiver class and its subclasses serve as “subscribers” in publish/subscribe (or pub/sub) communication mechanism implemented by Android application architecture.

Intent class and its subclasses serve as messages in message passing and publish/subscribe communication mechanisms implemented by Android application architecture.

We’ll discuss the communication mechanisms in another post later on.

Of course, Android application developer is not limited to just extending Android SDK classes. She can write her own classes as she wants. But all they will be just “helper classes” of sorts for those main classes derived from Android SDK classes.

Android manifest

Android manifest is another important part of Android application. The idea has been clearly inspired by Eclipse plug-in manifest.

Android manifest is an XML file and it serves several functions. Here is Google description of manifest functions:

  • It names the Java package for the application. The package name serves as a unique identifier for the application.
  • It describes the components of the application — the activities, services, broadcast receivers, and content providers that the application is composed of. It names the classes that implement each of the components and publishes their capabilities (for example, which Intent messages they can handle). These declarations let the Android system know what the components are and under what conditions they can be launched.
  • It determines which processes will host application components.
  • It declares which permissions the application must have in order to access protected parts of the API and interact with other applications.
  • It also declares the permissions that others are required to have in order to interact with the application’s components.
  • It lists the Instrumentation classes that provide profiling and other information as the application is running. These declarations are present in the manifest only while the application is being developed and tested; they’re removed before the application is published.
  • It declares the minimum level of the Android API that the application requires.
  • It lists the libraries that the application must be linked against.

Please note the second point. It means even if there is a class extending Activity, ContentProvider , BroadcastReceiver or Service within an application, the class can’t be used unless it is explicitly described in Android manifest.

Resources

Any modern GUI application technology uses resources in some form. Android applications are not an exception of the rule. They use following types of resources:

  • Pictures
  • GUI layouts (XML files)
  • Menu definitions (XML files)
  • Textual strings

The way resources are referenced by Android applications is somewhat unusual. Normally in Java resources are identified by strings. Such strings may contain e.g. a path and a name of a file containing the picture or an ID of particular string etc. The problem with such approach is that mistakes in those references could not be caught during code translation.

Let’s consider following example. A file named mybutton.png contains a picture for a button. A developer makes a mistake and types mybuton.png as a reference to the resource in the code instead of using the correct file name. As a result the code is instructed to use nonexistent resource, but the code will compile just fine. The mistake may be discovered only during testing (and may not be discovered at all).

Google folks came with an elegant solution to the problem. Android application build generates a special Java class named R (just a single letter name) for each package where classes use resources. This class has a number of static final data members. Each such data member is a reference to a particular resource and they are used in Android Java code for referencing resources. Due to this each mistake in referencing resources is discovered at compile time.

Files

Android applications use several different types of files:

  • “General purpose” files
  • Database files
  • Opaque Binary Blob (OBB) files (those represent encrypted file systems on its own that can be mounted for the application)
  • Cached files

Although ultimately they are all just Linux files, it makes sense to consider them as separate types of files since they are handled by different Android APIs and stored separately. There is also a distinction between files stored in internal device storage and in external device storage (the latter may be not present or disappear/appear at any moment).

APIs for working with the files are implemented by Context class which is an ancestor for Activity and Service classes. We already discussed this class here.

That’s all for now. In the next post we’ll discuss how different parts of Android applications communicate between them.

Read Full Post »

Older Posts »

Follow

Get every new post delivered to your Inbox.

Join 55 other followers