Data Management meeting held the 30/09/2003

Present: P.Charpentier, J.Closier, M.Frank, V.Garonne, 
         G.Kutznetsov, F.Loverre, S.Ponce, A.Tsaragorotzev

Minutes and action list.

Agenda:
=======
- Fixing time slots for future meetings     (Markus)
  The meetings will be every Tuesday at 9:30
  Markus will arrange a telephone conference
  for this time slot.
  === Action: Markus

- Bookkeeping
  - Metadata and job tracing
    Review current status                   (Sebastien)

    Sebastien gave an overview of the current system.
    Now all data accesses for the user interface 
    are implemented in JAVA. The code still needs
    some cleanup to be done by Francoise. Also
    the implementation to select datasets
    not only by reconstruction and simulation program,
    but also by e.g. stripping program, event tags etc.
    must still be done by Sebastien.
    Once the implementation is finished, the 
    JAVA code will move to the ORACLE web server,
    where Joel agreed to arrange the move.
    === Action Francoise, Sebastien, Joel

  - New ideas on retrieving the information (Andrei)

    The group around Andrei will investigate the use
    of the Alien bookkeeping system, which is based on a 
    hierarchical structure. The idea is to move 
    towards a standard ARDA interface.
    === Action Andrei

- Filecatalogs
  - Technology choice
    (EDG-RLS, Alien, POOL-MySQL-LHCb)       (Andrei)

    The file catalogue will not only have to be used
    to resolve physical file names from file identifiers (FIDs),
    but also to manage job distribution etc. Since this
    overlaps very much with what Alien intends to do,
    Andrei will investigate the possibilities of a possible
    implementation based on Alien. Any implementation must
    be based on an ARDA interface, which will have to be
    defined and endorsed by LCG. The file catalogue interfaces
    of POOL and ARDA will have to be compliant.
    === Action: CPPM/Andrei

- User analysis and simulation productions  (Andrei)
  - Workload management service(s)
    - Reprocessing and individual user 
      analysis
    - Simulation production 
  
    Andrei presented his ideas of a new work load 
    management system. The approach is very generic 
    and will also allow to manage user analysis.
    In this context the existing production system is 
    only one possible client of the work load manager.
    Ganga and others will share the work load manager.
    The development of this system will largely be done
    around the group of Andrei and a prototype will be
    available soon.
    === Action: CPPM/Andrei

  - New production tools under development  (Gennady)

    Gennady presented his ideas for the development 
    of implementing a modular data production
    editor based on the work load manager under development
    by Andrei. The editor is split into several 
    components, which deal with the development of
    modular processing steps, the construction
    of work flows and finally the incarnation of
    a work flow into a data production with
    many jobs as it was described in the data management
    paper. These tools, which will ease the work of
    the data production manager will be integrated
    into a DIRAC desktop environment for a coherent
    look and feel.

    This work was seen to be very important, because
    it will allow to also describe data reprocessing
    and data reduction tasks, which in the current 
    production system cannot be handled properly.
    The general impression was, that the plans
    of Gennady allow for a very flexible implementation,
    which would also allow to deal with Ganga user analysis
    jobs. How this can be achieved will have to be
    investigated.

    Philippe and Andrei were worried, that
    parts of this work would duplicate the efforts
    of the work load management system under development
    in Marseille. Hence Gennady and Andrei are invited 
    to coordinate their efforts and work together more
    closely. 
    === Action: Gennady, Andrei

  - Necessary changes to the existing       (All)
    production system


Cheers,

     Markus Frank