Using Globus certificates to authenticate to a remote rootd/proofd server
=========================================================================

To use Globus certificates for remote authentication, the Globus Tool Kit
should  be installed both in client and server machines and ROOT should be
compiled with  the relevant flags set (see below).

Globus Authentication has been tested for the moment only on Linux.

1) Globus Tool Kit (GTK) Version

   The interface has been developed using GTK version 2.2.3 and tested on
   machines with version 2.0, 2.0.21b, 2.4.3 and 3.2.1; it should work with
   versions 2.x and 3.x, at least until the C API's are kept ; however,
   vanilla versions 2.2.3 - 2.4.3 contain a bug which does not allow to 
   overload at the same time the defaults for the user certificate
   and key files. A patch has been provided by the Globus team (see Globus
   bugzilla #892) which is shipped with ROOT.

   If your system does not have GTK installed, you do not need to install
   all the GTK bundles but only the GSI one, i.e.

        globus-gsi-<GTK_version>-src_bundle.tar.gz

   Versions older than 2.1.2 and 3.x versions do not have a separate bundle
   for GSI: in such a case you need to install the 

        globus-resource-management-client-<GTK_version>-src_bundle.tar.gz

   If you plan to install binaries, the relevant code should be contained in
   the 

        globus-resource-management-client-<GTK_version>-<arch>-bin.tar.gz

   bundle.
   Please, refer to www.globus.org for any additional information about 
   installation and certificate acquisition. 

2) Compiling ROOT with Globus support

   To compile ROOT with Globus support make sure, *before* running 'configure',
   that the environment variable GLOBUS_LOCATION is defined and points to the
   location with the 'bin', 'lib' and 'include' dirs for the GTK; 'configure'
   should then pickup the correct paths. While it is definitely better to have
   GLOBUS_LOCATION defined correctly (especially on the client side), for
   compilation purposes only it is possible to force a specific directory for
   GTK running 'configure' with

       --with-globus-dir=<GTK_path>

   The 'configure' script checks if the used GTK needs the patch mentioned 
   above. While running configure you should get the following printed

 ...
Checking for globusdir ... no
Checking for GLOBUS_LOCATION ... <GLBSLOC>
Checking for openssl/x509.h ... <GLBSLOC>/include/<flavour>
Checking for openssl/pem.h ... <GLBSLOC>/include/<flavour>
Checking for globus_gss_assist.h ... <GLBSLOC>/include/<flavour>
Checking for globus_gss_common.h ... <GLBSLOC>/include/<flavour>
Checking for globus_gsi_credential.h ... <GLBSLOC>/include/<flavour>
Checking for gsi version ... yes
Checking for libglobus_gss_assist_<flavour> ... <GLBSLOC>/lib
Checking for libglobus_gssapi_gsi_<flavour> ... <GLBSLOC>/lib
Checking for libssl_<flavour> ... <GLBSLOC>/lib
Checking for libcrypto_<flavour> ... <GLBSLOC>/lib
Checking for libglobus_gsi_credential_<flavour> ... <GLBSLOC>/lib
Checking for globus-user-env.sh ... <GLBSLOC>/etc
Checking for grid-proxy-init ... <GLBSLOC>/bin
Checking pkg_data_<flavour>_rtl.gpt ... <GLBSLOC>/etc/globus_packages/globus_gsi_credential
Checking for of globus patch ... yes, version <globus_gsi_credential version>
Checking for patch version consistency ... yes
Checking for globus_gsi_system_config.h ... <GLBSLOC>/include/<flavour>
Checking for globus_gsi_cert_utils.h ... <GLBSLOC>/include/<flavour>
Applying experimental patch to globus_gsi_credential.c
 ...

   where <flavour> is either gcc32 or gcc32dbg, and <GLBSLOC>
   is the env $GLOBUS_LOCATION. 
   The last 3 lines appear only if the patch is needed and activated.

   If globus_gsi_credential.c does not compile correctly, it may be that
   the patch version found by configure is not correct. Check in the 
   globusauth/src/patches/globus_gsi_credentials/README if the printed
   <globus_gsi_credential version> matches the one expected for your 
   GTK version (ask the latter to your site administrator if you do not
   it). If the version is wrong you should copy the right patched version
   of the globus_gsi_credentials.c file

cd  globusauth/src
cp -rp patches/globus_gsi_credentials/<right version>/globus_gsi_credentials.c .
cd ../..

   and substitute the right version for the wrong one in the variable 
   GLBPATCHFLAGS defined in config/Makefile.config; re-compile *without* 
   running configure. If the problem persists, mailto: gerardo.ganis@cern.ch.

3) Before running, after compilation/installation

   3.1) On the CLIENT side

        Make sure that valid certificates are in the correct places and that
        valid credentials can be initialized; this can be tested by running

               grid-proxy-init

        and entering the pass phrase; you may have to source the relevant shell
        file to have this command recognized:

           source $GLOBUS_LOCATION/etc/globus-user-env.csh
        or
           source $GLOBUS_LOCATION/etc/globus-user-env.sh .

        If you are sure about your capabilities to get valid credentials, you
        don't need to initialize them outside the root session, for ROOT checks
        the credentials and prompts for the pass phrase if they are missing or
        invalid.

        There are three files/dirs relevant for proxy initialization:

        a) user certificate file; default is $HOME/.globus/usercert.pem ; can
           be  changed by setting the variable "X509_USER_CERT" to the full
           path

        b) user private key file; default is $HOME/.globus/userkey.pem ; can
           be  changed by setting the variable "X509_USER_KEY" to the full path

        c) directory with certificates of the known Certificate Authorities;
           default is /etc/grid-security/certificates ; can be changed by
           setting the variable "X509_CERT_DIR" to the full path.

        All this variables can be changed by the user before running root or
        during a root session. Globus authentication can be requested by:

        1) Setting

                Rootd.Authentication   3
                Proofd.Authentication   3

           in the $HOME/.rootrc file; other parameters can be changed at this
           level:

          - the proxy duration, in hours

                Globus.ProxyDuration :    <hours>

          - the number of bits to be used for the key (should be a power of 2)

                Globus.ProxyKeyBits:      1024

          - a Globus login defining the relevant files mentioned above:

                Globus.Login:             cd:~/.globus cf:usercert.pem  \
                                          kf:userkey.pem  ad:certificates

                with the meaning of the different keys being:

                "cd"    defines the directory with the user certificate and
                        private key; default: $HOME/.globus
                "cf"    user certificate file (in the dir specified by "cd");
                        default: usercert.pem
                "kf"    user key file (in the dir specified by "cd");
                        default: userkey.pem
                "ad"    directory with certificates of the known Certificate
                        Authorities; default: /etc/grid-security/certificates

                Both "cd" and "ad" can be given as absolute paths (starting
                with '/'), relative to  $HOME (starting with ~/) or relative to
                $HOME/.globus (in the other cases).

        2) Specifying an entry in .authrootrc (see README.AUTH for more details)

           This allows to be host and user specific and to initialize proxies
           for two or more different users in the same session.

        3) During a ROOT session by creating (or modifying) a THostAuth
           instantiation (see README.AUTH for more details)

        Upon successful authentication the user is logged on remotely with the
        user name  specified in the gridmap file on the remote server; if there
        is no entry is found in  the gridmap corresponding to the client DN, an
        attempt is made to guess it from the  CN part of the DN.


   3.2) On the SERVER side

        The servers daemons (rootd/proofd/TServerSocket) need a valid service-type 
        certificate and key pair to handshake mutual authentication with the
        client.

        By default, the search for a such a certificate/key pair is done in
        the following order:

          - ROOT specific {certificate,key} under /etc/grid-security/root;
            file names: rootcert.pem, rootkey.pem ;

          - host {certificate,key} under /etc/grid-security;
            file names: hostcert.pem, hostkey.pem ;

          - user proxy {certificate,key} under /tmp, file name: x509up_<uid>;

        The reason to have the last one is to allow unprivileged users to
        run GSI authentication for their own local server. Make sure that
        the time validity of the proxy is long enough for the purposes of 
        the server. Proxies can initialized for any time duration (within
        the time validity of the certificate) with
             
            grid-proxy-init -valid <large_number_of_hours>:0

        The proxies *must be* initialized before the daemon tries to use
        them, i.e. there is no prompt for proxy initialization as on the 
        client side.

        Any other service certificate may be used in place of the ROOT one:
        the important point is that the key file can be read by the daemon.
        To create a request for a ROOT specific certificate use

        grid-cert-request -service root -host <machine_FQDN>

        The certificate request will be placed under /etc/grid-security/root .
        If you do not have write permission of /etc/grid-security, you can
        redirect grid-cert-request to write on a different directory using
        '-dir <alternative_directory'.

        Alternative location(s) for the certificates to use can be specified
        by means of the hostcert.conf file. By default this file is searched
        for in /etc/root/hostcert.conf or in $ROOTSYS/etc/hostcert.conf .
        However, daemon servers accept the argument
	    
                        -C <server_globus_conf_file>

        to specify an alternative location; for daemons based on TServerSocket
        the ROOT environment variable 

                        SrvAuth.HostCert <server_globus_conf_file>

        is provided.
        The hostcert.conf contains record lines specifying
	    
        <certificates_dir> <host_cert_file> <host_cert_key> <grid_mapfile>
	    
        See the header of $ROOTSYS/etc/hostcert.conf for any details about
        the syntax and sue of the file; as an example:
	    
< bof >
# This is an example of hostcert.conf to use the ldap service certificate
  * /etc/grid-security/ldap/ldapcert.pem /etc/grid-security/ldap/ldapkey.pem *
< eof >

        This file provides also a convenient way to specify certificate settings
        corresponding to different Certificates Authorities and potentially
        to different DN-to-UserName mappings. When a request for globus
        authentication arrives, the daemons look among their own certificates
        if there is one issued by the same CA which has issued the client
        certificate; if the search is successful they communicate the related
        subject name to the client and setenv the relevant variables using to
        the chosen configuration set.

        The hostcert.conf file can also be used to specify non-standard locations
        for the user proxy files.


        .2.1) Using user proxies when running as 'root' (or with su privileges)

        If a valid host certificate is not available, the server administrator 
        can still decide to run as 'root' and use the valid proxies of a user
        registered on the machine to authenticate a client. 
        To activate this, the user certificate and key location should be
        specified in the relevant hostcert.conf file, e.g.

        < bof >
        # This is an example of hostcert.conf ...
         * /home/<user>/.globus/usercert.pem /home/<user>/.globus/userkey.pem *
        < eof >

        The program will then look for the proxy file associated with the owner
        of the certificate and key files and use that to acquire the credentials
        needed for authentication; as above, the proxies must be initialized
        before the relevant software tries to use them.

4) Trying out

   Trying to access a remote file should give this on the client side (assuming
   globus credentials have not been initialized before):

root [1] TFile *f1 = TFile::Open("root://<remote_FQDN>/test.root","read")
Your identity: /O=Grid/OU=GlobusTest/OU=simpleCA-arthux.cern.ch/OU=cern.ch/CN=qwerty
Enter GRID pass phrase for this identity:
Creating proxy ....................... Done
Your proxy is valid until Wed May 28 02:46:12 2003
root [2]

   In the /var/log/messages on the server you should get something like ( with -d 3 in 'rootd')

May 27 12:49:46 pcepsft43 rootd[24031]: RootdLoop: kind:2033 -- buf:'11820 -1 2 4 None' (len:17) -- Auth:0
May 27 12:49:46 pcepsft43 rootd[24031]: RpdDefaultAuthAllow: default list of secure methods available:  4 1 2 3
May 27 12:49:46 pcepsft43 rootd[24031]: RpdGlobusAuth: user: /O=Grid/OU=GlobusTest/OU=simpleCA-arthux.cern.ch/OU=cern.ch/CN=qwerty   authenticated
May 27 12:49:46 pcepsft43 rootd[24031]: RpdGlobusAuth: logging as qwerty
May 27 12:49:46 pcepsft43 rootd[24031]: RootdLogin: user qwerty authenticated (OffSet: -1)
May 27 12:49:46 pcepsft43 rootd[24031]: RootdLoop: kind:2004 -- buf:'/test.root read' (len:10) -- Auth:1

   The first authentication to a (remotehost,username) entity may be perceived
   as slow;  this is because a lot of exchanges are need to establish the
   security context; however,  the security context is kept until is valid, so
   subsequent access to the same entity  will be much faster (if this is not
   so, make sure that you do not have set the 'reuse'  flag to '0' or 'no'; see
   README.AUTH).

   In a PROOF cluster, if the master requires Globus credentials for
   authentication  to slaves, these are automatically transmitted using a
   shared memory.

5) If it does not work ...

   Some tips:

   .1) make sure that local and remote times are synchronized within 5 minutes
   .2) if GlobusAuthenticate is not found and you are using /etc/ld.so.conf for
       shared libraries you may need to run /sbin/ldconfig -v after compilation.
   .3) Make sure that remote node accepts Globus authentication from your
       local host
   .4) Check location of certificate and private key files and certificate
       directories; make sure they can be read by the root process
   .5) If you get the error message:

        init.c:497: globus_gss_assist_init_sec_context_async: 
                    Error during context initialization
        init_sec_context.c:187: gss_init_sec_context: 
                                Unable to verify remote side's credentials
        globus_i_gsi_gss_utils.c:898: globus_i_gsi_gss_handshake: 
                    SSLv3 handshake problems: Couldn't do ssl handshake
        OpenSSL Error: s3_clnt.c:836: in library: SSL routines, 
                    function SSL3_GET_SERVER_CERTIFICATE: certificate verify failed
        globus_gsi_callback.c:349: globus_i_gsi_callback_handshake_callback: 
                    Could not verify credential
        globus_gsi_callback.c:435: globus_i_gsi_callback_cred_verify: 
                    Could not verify credential: 
                    self signed certificate in certificate chain (error code 7)

       your certificate issuer is not recognized as trusted.
       Most likely this means that its CA certificate has not been found. 
       Either you don't have it or one of the followings holds:
 
        o your $HOME/.globus/certificates does not exist or is empty
        o your X509_CERT_DIR is either undefined or points to the wrong
          directory

       Solution: you should localize the certificate of the CA issuing your 
       certificate: this is in the form <HASH>.0 and a possible alternative
       location in /etc/grid-securiy/certificates; you can check the content
       using the openssl command:

          openssl x509 -in <HASH>.0 -text

       Do the same with your $HOME/.globus/usercert.pem and make sure the 
       Issuer is the same. 
       Once you have localized the directory with your CA certificate, setenv
       X509_CERT_DIR to point to that directory, for example

          setenv X509_CERT_DIR /etc/grid-securiy/certificates

       (or export X509_CERT_DIR=/etc/grid-securiy/certificates for a bash shell). 

   .6) If ROOT has been built on a different machine, make sure that the correct
       (if any) GTK patch has been applied; a version mismatch may cause very 
       strange failures, quite difficult to debug / identify.
        
   .7) mailto: gerardo.ganis@cern.ch.

--------------------------------------------------------------------------------------
Last update: February 22, 2005