Standardisation, benchmarking and comparision of different experimental systems

Large-scale functional genomics experiments are becoming more widespread to take advantage of sequence databases. These experiments rely mostly on new protocols or technologies that are still developed and practised by only a relatively small number of laboratories. However, similar biological questions may be addressed by various techniques and results may not be comparable without clear definitions, creating an urgent need for standardised criteria for different technological approaches. In addition, it is essential to provide a framework for comparison of results obtained with the same underlying technology in different laboratories. As an example, different methods have been developed for production of large scale cDNA microarrays for gene expression experiments. The use of different array systems has created a particular need for comparison of results and benchmarking between different laboratories. A publicly available database for storage and analysis of microarray data is now needed. This subject will become increasingly important, especially as the scientific community plans to have biological databases dealing with heterogeneous data to be exploited as a guide to define new biological experiments.

Similarly, analysis of the complex heterogeneous data needs to be done in a way which allows results to be related meaningfully. This can only be achieved in a multidisciplinary approach in cooperation with all the major laboratories involved in the different levels of functional genomics studies. Bioinformatics and database developments have to be in close collaboration with the experimental research teams in order to produce databases which can collect genomic data and make them accessible to the scientific community, in a similar way that sequence databases do today for sequence data. The success of functional genomics efforts will depend to a great extent on parallel approaches of standardisation both on the bioinformatics as well on the biological side.

The objectives of standardisation and benchmarking can only be achieved through extensive discussions between relevant experts and by distributing specialised tasks among the members of the programme network. Materials such as high density DNA filters or glass slides containing minimal clone sets, protein expression libraries or two-hybrid systems are very costly and resource intensive to produce, requiring in most cases expensive equipment and experienced staff. It would therefore be advantageous to have resource centres to produce such materials in a standardised way. This would produce the benefits that, firstly, all laboratories would be able to use high quality and affordable material for their research and, secondly, it would be possible to compare experimental results more easily due to the common source materials used. This could be applied similarly to the successful use of common high quality reference libraries for mapping and sequencing projects (e.g. the CEPH YAC library, Pieter de Jong´s BAC and PAC libraries, the chromosome specific cosmid libraries from LLNL and LANL, and the IMAGE clone collection) and specialised but linked databases (e.g. GDB, RZPD, EMBL, Genbank, Swissprot, dbest and others) to store data produced with these materials. The agreement to standardise and benchmark experimental techniques and analysis tools, as well as the access to standardised reference material, are the prerequisites for development of major functional genomics databases. Standardisation of mutational analysis is particularly important for the delivery of the new paradigm in healthcare, to enable reliable definition of human genotypes.

An ultimate goal of the programme would be to standardise all relevant items involved in data production and to develop databases which allow groups working in similar areas to store their data in a common 'virtual centre' and thus make them accessible to other scientists. It is essential to ensure that all data can be traced to the originating laboratory; in this way it will be possible for individual scientists to resolve conflicting data by direct comparison of methods and experimental systems. It is intended that the programme will act as a forum for standardisation of functional genomics technologies in Europe. This includes the development of database schemas as well as user-friendly graphical interfaces (preferably using the WWW) to submit, update, search, display and analyse the data and to link them to existing databases storing mapping, sequence and other structural information. The ambition would be to link hundreds of European laboratories that would agree on standard formalised presentation of results. Global integration of standards will be a major objective.

Contacts within the programme

Pierre Legrain
Chris Mundy