Introduction

The basic principle of optimization is not to optimize. This sounds contradictory, but this means that you should not try to optimize code until you know it is really necessary to optimize it, and how to optimize it. Until you realize optimization is required, you should respect readability and elegancy.

Profiling

The first thing you should do is to profile your code. Quite often, a bad performance is derived from one or a few points of your code, so-called bottlenecks. Even if you improve a piece of your code significantly, if that part is not a bottleneck, you see no performance gain as a whole. So you must devote yourself on figuring out bottlenecks and solving them.

The most useful tool in profiling Zope applications is ZopeProfiler. ZopeProfiler allows you to profile code both in the level of Zope and in the level of Python, and you can turn on and off profiling at run time.

Using ZopeProfiler is easy. Download the product, extract the tarball in your instance home, and restart the Zope server. Then, you will see the interface of ZopeProfiler in the Control Panel. When you enable the ZopeProfiler, it begins profiling every activity in the Zope server. When you disable it, it shows statistics during the profiling time. Since ZopeProfiler puts information on the memory, you should not enable it too long. Otherwise, Zope will eat all of your memory and crash.

To profile your code correctly, you should:

1. stop all activities but what you want to check. For this reason, it is better to use your own Zope server.
2. stop tic and distribute, if you use portal_activities, unless your intention is to check the performance in an usual environment.
3. disable portal_activities (e.g. renaming it to something else), if you want to check activity processes as well.
4. execute the same thing many times or use a task big enough. Because profiling is affected by random noises, an attempt does not produce the same result as another attempt. To reduce the effect of random factors, it is necessary to use long computations. In particular, caching affects the system very much.
5. not assume that the result is the same as without ZopeProfiler. Because profiling adds a big overhead for tracing the code, this is not completely the same as a real run. Usually, profiling makes all operations very slow, but not equally slow. You should use the result as an estimate.

Algorithms

Choosing good algorithms is the best. For example, if you can make O(n^2) to O(n), the performance gain would be extremely significant. Remind yourself of the theory of computational complexity. You should have leant it in your school... Otherwise, find a good web site or buy a good textbook.

Python

There are some tips on Python-specific optimizations. Look at Python Speed and Python Performance Tips.

Installing Psyco is very simple but very powerful. But, note that Psyco is available only on IA-32, and it cannot compile code if you use map or filter. This is a good reason why we should use list comprehension.

Z SQL Methods

Z SQL Methods are frequently bottlenecks. This is sometimes due to DTML expressions and sometimes due to MySQL. The principle is that fewer SQL queries are better, fewer DTML expresions are better.

Look at MySQL Optimization.

Caching

ERP5 provides a generic way of caching methods. It is called CachedMethod. If you must invoke a heavy method many times, caching may be useful for you. You must, however, take care of sub-effects of caching. Caching means that the user sees a bit older data for a while. This may lead to an inconsitent state, if you do not invalidate the cache appropriately.

Conflicts

You should avoid conflicts. Conflicts occur, when multiple transactions try to access the same object, and one or more of them tries to change the object. When conflicts happen, transactions are rolled back, and restarted from the beginning. This reduces the performance radically in a multi-user environment. (Where to look at more information? ZODB Development Wiki ?)

Activities

If there is no way to improve the performance only at the level of software, another way is to use many computers. Using portal_activities, you can distribute method calls very easily to a large number of computers.