CAMS McClear automatic access

Last update: May 2017

    With wget command - With an XML request

CAMS McClear may be accessed in an automated way.
The automatic access is for free. It is limited to registered users.

CAMS McClear follows the WPS (Web Processing Service) standard that defines how a computer can invoke a web service, without human-machine interface.  This way of doing is a standard in Earth Observation and Digital Geography domains.

You have 2 ways to call CAMS McClear:

  • an easy way using the wget command
  • a more complete way using an XML request

1_ With the wget command

Build your first wget request in 5 minutes

  1. This service is limited to registered users.
  2. Build your first request as follows, using the login that you entered for your registration to the SoDa website. Let's admit that you are in a Windows environment. Copy and paste the following line in a text file, and rename it with the .bat extension (request over the city of Carpentras, France (44.083°, 5.059°), from 2017-01-01 to 2017-01-05, 15 min time step, login address: myemailaddress@mycompany.fr):

Request (Windows, write the following command in a single line): wget -O CAMS-McClear_Carpentras_2017-01-01_2017-01-05.csv "http://www.soda-is.com/service/wps? ...
Service=WPS&Request=Execute&Identifier=get_mcclear&version=1.0.0&DataInputs=latitude=44.083;longitude=5.059;
altitude=-999;date_begin=2017-01-01;date_end=2017-01-05;time_ref=UT;summarization=PT15M;
username=myemailaddress%%2540mycompany.com&RawDataOutput=irradiation"
NB: you should use only one % on a Windows command terminal, an internet browser or on Unix, but use a double %% ...
in a .bat executable file on Windows.

  1. Download wget.exe in the same directory of your .bat file, and launch your command by double clicking on your .bat file.

For more details on Wget (Unix / Windows, automatic shift to the mirror website, retrieve automatically the data for the previous month / day...), please refer to the page dedicated to the automatic access to HelioClim-3.

Build wget requests

Request (Unix and Windows, write the following command in a single line):
wget -O output.csv "http://server/service/wps?Service=WPS&Request=Execute&Identifier=get_mcclear ...
&version=1.0.0&DataInputs=latitude=latitude;longitude=longitude;altitude=altitude;date_begin=yyyy-mm-dd; ...
date_end=yyyy-mm-dd;time_ref=time-reference;summarization=time-step;username=email-address ...
&RawDataOutput=irradiation

With (inputs):

  • server: "www.soda-is.com" (main server) or "pro.soda-is.com" (backup-mirror server)
  • latitude and longitude: in degrees, with at least 3 digits after comma
  • altitude: in meters. Set "elevation=-999" to let SoDa get the elevation from NASA SRTM database
  • yyyy-mm-dd: set the first and the last days of the request
  • time_step: "PT15M" (minutes), "PT01H" (hourly), "P01D" (daily), "P01M" (monthly)
  • time_reference: "UT" (universal time) or "TST" (True Solar Time)
  • email-address: your email address used to generate your SoDa account, by replacing the "@" by "%%2540" if this command is in a Windows executable .bat file, and by "%2540" if on a Windows command terminal, an internet browser or on Unix

For "summarization=PT01M" only, a verbose mode is available  by adding "verbose=true" to the DataInputs. Example in a .bat file on Windows (write the command in a single line):
wget -O CAMS-McClear_Carpentras_2017-01-01_2017-01-05.csv "http://www.soda-is.com/service/wps?
Service=WPS&Request=Execute&Identifier=get_mcclear&version=1.0.0&DataInputs=latitude=44.083;longitude=5.059;
altitude=-999;date_begin=2017-01-01;date_end=2017-01-05;time_ref=UT;summarization=PT01M;
username=myemailaddress%%2540mycompagny.com;verbose=true&RawDataOutput=irradiation"

Outputs:

Identical to the Web service available online

  1. Observation period (ISO 8601)
  2. TOA. Irradiation on horizontal plane at the top of atmosphere (Wh/m2)
  3. Clear sky GHI. Clear sky global irradiation on horizontal plane at ground level (Wh/m2)
  4. Clear sky BHI. Clear sky beam irradiation on horizontal plane at ground level (Wh/m2)
  5. Clear sky DHI. Clear sky diffuse irradiation on horizontal plane at ground level (Wh/m2)
  6. Clear sky BNI. Clear sky beam irradiation on mobile plane following the sun at normal incidence (Wh/m2)

In verbose mode (1 minute summarization only!), additional columns are:

  1. sza. Solar zenith angle for the middle of the summarization (deg)
  2. atm. Atmospheric profile code: afglus=U.S. standard afglt=tropical afglms=midlatitude summer afglmw=midlatitude winter afglss=subarctic summer afglsw=subarctic winter
  3. tco3. Total column content of ozone (Dobson unit)
  4. tcwv. Total column content of water vapour (kg/m2)
  5. AOD BC. Partial aerosol optical depth at 550 nm for black carbon
  6. AOD DU. Partial aerosol optical depth at 550 nm for dust
  7. AOD SS. Partial aerosol optical depth at 550 nm for sea salt
  8. AOD OR. Partial aerosol optical depth at 550 nm for organic matter
  9. AOD SU. Partial aerosol optical depth at 550 nm for sulphate
  10. AOD 550. Aerosol optical depth at 550 nm
  11. AOD 1240. Aerosol optical depth at 1240 nm
  12. alpha. Angstroem coefficient for aerosol
  13. Aerosol type. Type of aerosol: -1=no value 5=urban 7=continental clean 8=continental polluted 9=continental average 10=maritime clean 11= maritime polluted 12=maritime tropical 13=antarctic 14=desert
  14. fiso. MODIS-like BRDF parameter fiso
  15. fvol. MODIS-like BRDF parameter fvol
  16. fgeo. MODIS-like BRDF parameter
  17. albedo. Ground albedo

Warning! It is strongly recommanded to avoid the launch of parallel requests, using the "&" at the end of each wget request line. This would endanger our system.

2_ With an XML request

CAMS Radiation Service follows the WPS (Web Processing Service) standard that defines how a computer can invoke a web service, without human-machine interface.  This way of doing is a standard in Earth Observation and Digital Geography domains and offers many other advantages over a text-interface like the other services on SoDa web site. In particular, anyone can create a portal invoking these services by calling the WPS that we are offering. It greatly helps the dissemination.

These services are described in the catalog
www.soda-is.com/service/wps?SERVICE=WPS&request=GetCapabilities
that reveals the get_mcclear operations.

If you are using an appropriate library, it is fairly easy to develop the application.
Otherwise, you may try to find one. There are plenty, for example google "library execute wps ogc".

The call to DescribeProcess provides a description of the inputs and outputs of each service (that you discover in the result of the call to GetCapabilities):
www.soda-is.com/service/wps?SERVICE=WPS&Request=DescribeProcess&Identifier=get_mcclear&version=1.0.0

How to do it by hand (or in an application not calling specific libraries)?
The request will have the following form
curl -o output.xml -v -d @test_mcclear_params.xml -H "Content-Type: text/xml" http://www.soda-is.com/service/wps
where "output.xml" is the output file that contains the URL of the result (here a csv file) and "test_mcclear_params.xml" is the input file.
The input file contains the inputs (obviously...). Most of them have clear names and are exactly those which are requested on the human interface at soda-pro.com.
If choice is offered, e.g. "summarization", the various options are provided by the call to DescribeProcess. The "username" is the email registered in soda-pro.com.

- If this tutorial is not appropriate or incomplete, feel free to ** and contribute to the improvement of this page. Any feedback is welcome -