Release Notes CAMS

Improved version of SoDa/CAMS load balancer

This upgrade introduces an improved version of SoDa/CAMS load balancer. The load balancer can process more requests at the same time and has a better detection of failures.
We invite you to call CAMS requests on https://api.soda-solardata.com server instead of https://www.soda-is.com or https://www.soda-is.com.

Incomplete error message for CAMS request without registered email

Fixed error message for CAMS request without registered email (used to be truncated)

CAMS server crash in 2024-01 due to burst of traffic

Fixed crash in 2024-01 due to many CAMS requests at the same time

CAMS requests through load balancer are failing randomly

Fixed random bug in CAMS requests through load balancer for new CAMS users

Lock error in CAMS services

Fixed random error in CAMS services when many requests were executed at the same time

#1088

SSL certificate of https://solar.atmosphere.copernicus.eu obsolete

Access to https://solar.atmosphere.copernicus.eu has being impossible since mid-February due to an obsolete SSL certificate. This problem is now fixed.

#1084

Crash for McClear request with invalid latitude or longitude

Fixed bug that used to crash McClear request if latitude is not between -90 and 90 deg, or longitude is not between 0 and 360 deg

CAMS Radiation Service for HIMAWARI field of view

On 2023-10-17, the CAMS Radiation Service for HIMAWARI satellite field of view will open officially.

CAMS Radiation Service API is unchanged. See the documentation at https://www.soda-pro.com/help/cams-services/cams-radiation-service/automatic-access 

Feel free to contact soda@transvalor.com for more details.

#1035

New error messages for CAMS Radiation and McClear services

In order to support HIMAWARI satellite, SoDa will switch on 2023-10-17 to a new version of CAMS Radiation and McClear services. This will be mostly transparent for users, except error messages that are modified (new XML file format).

Please adapt your error handling code if you use CAMS Radiation and McClear services API.

See the API documentation at:
- https://www.soda-pro.com/help/cams-services/cams-radiation-service/automatic-access 
- https://www.soda-pro.com/help/cams-services/cams-mcclear-service/automatic-access

Please adapt your error handling code if you use CAMS Radiation and McClear services API

Beta test: CAMS Radiation Service for HIMAWARI field of view

On 2023-07-03, the CAMS Radiation Service for HIMAWARI satellite field of view will open in beta test.

CAMS Radiation Service API is unchanged. See the documentation at https://www.soda-pro.com/help/cams-services/cams-radiation-service/automatic-access 

Feel free to contact soda@transvalor.com for more details.

#1035

New error messages for CAMS Radiation and McClear services

In order to support HIMAWARI satellite, SoDa will switch on 2023-07-03 to new servers hosting CAMS Radiation and McClear services. This will be mostly transparent for end users, except error messages that are modified.

To adapt your acquisition routine, you may already check new messages by using the test server http://vhost5.soda-is.com instead of the classic server https://api.soda-solardata.com.

See the complete documentation at:
- https://www.soda-pro.com/help/cams-services/cams-radiation-service/automatic-access 
- https://www.soda-pro.com/help/cams-services/cams-mcclear-service/automatic-access

Please adapt your code

New improved version of CAMS Radiation Service all sky model. This version 4.6 uses the new version 48r1 of CAMS IFS numerical weather introduced on 2023-06-27.

Please note that a new column SOAOD550 has been added to the csv output format of the Expert mode.

In CAMS Radiation Service verbose mode, cloud type was 5=low-level cloud on clear sky instants. It should be 0.

New improved version of McClear clear sky model. This version 3.6 uses the new version 48r1 of CAMS IFS numerical weather introduced on 2023-06-27.

Data prior to 2023-06-27 are unchanged.

Please note that a new column SOAOD550 has been added to the csv output format of the Expert mode.

https://api.soda-solardata.com

A load balancer has been set up, it will ease your usage of the CAMS services :

https://api.soda-solardata.com

The examples of scripts are updated (see url : https://www.soda-pro.com/help/automatic-access/examples-of-scripts )

#990

McClear monthly request in TST time reference

McClear now accept requests in TST for monthly time step.

#520

2nd APOLLO_NG processing chain

CAMS Radiation Service uses internally APOLLO_NG model from DLR to compute clouds opacity. Since 10/26/2022, CAMS Radiation Service uses a second APOLLO_NG production chain to improve redundancy.

New improved version of CAMS Radiation Service all sky model. This version 4.5 uses a better clear sky model (McClear 3.5, see below) and no bias correction anymore.

New improved version of McClear clear sky model. This version 3.5 reduces the bias wrt ground measurements thanks to CAMS IFS Reanalysis data provided by ECMWF.

The CAMS-AOD service, delivers time series of Aerosol Optical Depth provided by ECMWF. This service is available from 2004 to 2021-06-30.

The end date has been fixed in CAMS-AOD form https://www.soda-pro.com/web-services/atmosphere/cams-aod

#854

On 2022-06-15, SoDa web services server http://www.soda-is.com​​​​​​​ will switch to https.
If you use SoDa and CAMS services only via https://www.soda-pro.com web site, you are fine.
If you use SoDa/CAMS services automatic access, i.e. if you directly call SoDa/CAMS web services, you will have to adapt your code.

On 2022-06-15:

• http://www.soda-is.com will remain a regular http server (without redirection to https)
• https://www.soda-is.com will be a new https server

We will keep both http://www.soda-is.com and https://www.soda-is.com running for several months, and you will have plenty of time to adapt your code to https://www.soda-is.com after the upgrade.

#848

The CAMS Radiation and McClear services are funded by the CAMS project of the European Union. The EU requires addditional information about CAMS users: country and type of company.

#854

On 2022-02-09, http://www.soda-pro.com​​​​​​​ and http://solar.atmosphere.copernicus.eu will switch to https. This change should be transparent to SoDa users.

Please note that SoDa web services server http://www.soda-is.com will not switch to https on 2022-02-09. This is planned for the next SoDa upgrade, in March 2022.

#857
 

www.soda-pro.com web site is implemented in a Content Management System named Liferay.

On 2021-11-03, www.soda-pro.com switches to Liferay 7.3. This change should be transparent to CAMS users.

This is a technical step necessary to the future switch to https.

#860

In compliance with European Union General Data Protection Regulation (GDPR), SoDa has always limited the collection of personal data to what is strictly necessary.

This point is now documented on www.soda-pro.com, incuding the cookies used by the web site.

We faced a gap in APOLLO data from 2021-10-20 to 2021-10-28. This gap is fixed since 2021-10-31.

#900

McClear clear-sky model uses a description of the atmosphere provided by CAMS IFS meteo model.

ECMWF upgraded CAMS IFS meteo model to version 47r3 on 2021-10-12. This will improve slightly McClear clear-sky model (and Helioclim-3 v5 and CAMS Radiation models that use McClear) for future requests.

Note that data before 2021-10-12 are not modified.

#235
#813
#867
 

Since 2013, CAMS Radiation service uses a cloud model named APOLLO provided by the DLR research lab. The version 4.0 of CAMS Radiation Service includes a new version APOLLO Next Generation that improves cloud detection using fuzzy logic.

CAMS Radiation Service 4.0 fixes the MSG satellite projection before 12/2017.

CAMS Radiation Service 4.0 includes also a brand new global correction that minimizes the error with respect to ground measurements.

#690

McClear clear-sky model uses a description of the atmosphere provided by CAMS IFS meteo model.

During 07/2019 - 09/2020 period, the version 46r1 of CAMS IFS did not evaluate accurately the amount of dust in the atmosphere. This lowered the quality of McClear in desertic regions.

This bug is fixed in CAMS IFS since version 47r1 in October 2020.

Since 2021-06-28, we also fixed the 07/2019 - 09/2020 period.
As CAMS Radiation Service uses McClear, this fix also applies to this service.

We faced a gap in APOLLO data from 2021-06-12 to 2021-06-16. This gap is fixed since 2021-06-17.

CAMS Radiation service uses a cloud model named Apollo provided by the DLR research lab.

We faced a gap in Apollo data from 2020-07-16 to 2020-07-27. This gap is fixed since 2020-08-06.

#775

#728
#749
#752

SoDa server at MINES ParisTech premises (hosting www.soda-pro.com and www.soda-is.com) faces random failures since 2019-11-13.

The problem is due to SoDa users doing (too many) requests in parallel.

This update implements a partial fix.

#741

Maintenance operation at the DLR premises from 2019-12-27 to 2019-12-31

CAMS Radiation Service has not been updated from 2019-12-27 to 2019-12-31 due to a large maintenance operation at the DLR premises. The DLR is the research lab that operates Apollo, the cloud model used by CAMS Radiation Service.

The missing data have been recomputed and the gap is filled since 2020-01-08.

Please redo CAMS radiation requests for period 2019-12-27 to 2019-12-31

New version of CAMS Radiation volumes over Europe and Africa with 2017 and 2018

A new version of CAMS Radiation volumes over Europe and Africa is available. We have added 2017 and 2018.

See http://www.soda-pro.com/help/cams-services/cams-radiation-service/download-europe-volume and http://www.soda-pro.com/help/cams-services/cams-radiation-service/download-africa-volume

#706

Cams/McClear monitoring was on error for un-requested too long period.

#701

#702

Apollo data are missing

Apollo data are missing on

- 2019-07-04, 2019-07-27 and 2019-07-28

- 2019-08-12 and 2019-08-13

#707

#711

#716

An electrical maintenance and a network maintenance occured on SoDa main server (www.soda-pro.com) on 2019-08-13. Unfortunately, this maintenance created intermittent network failures on 2019-08-14 and from 2019-08-29 to 2019-09-03. This problem is fixed since 2019-09-03.

#690

McClear, CAMS Radiation and HelioClim-3 v5 use the modelisation of the atmosphere computed by CAMS IFS meteorological model. This model will be upgraded to version 46r1 on 2019-07-09. The main impact to McClear, CAMS Radiation and HelioClim-3 v5 will be the new modelisation of nitrate and ammonium in the atmosphere. See details at https://confluence.ecmwf.int/display/COPSRV/Implementation+of+IFS+cycle+46r1

2019-07-03 upgrade of www.soda-pro.com provides the compatibility of McClear, CAMS Radiation and HelioClim-3 v5 with CAMS IFS upgrade 46r1.

McClear, CAMS Radiation and HelioClim-3 v5 output values will be unchanged before 2019-07-09. After this date, these models error will be slightly reduced.

Please note the addition of nitrate and ammonium to the output formats of McClear and CAMS Radiation (mode "verbose" only) and to the output format of CAMS AOD service.

#673

CAMS radiation cloud data were not processed 2019-04-01 and 2019-04-02. This is now fixed

#669

CAMS Radiation web service is valid in Meteosat 0 deg satellite disk only. The service used to return values outside of this disk.
This is now fixed.

Integration of CAMS Radiation and McClear services into CAMS Atmospheric Data Store

CAMS template 12/2018

CAMS Radiation and McClear .csv requests contain now a new header line with the computation date

In case of McClear or CAMS Radiation monthly TST requests values used to be undefined (NaN).

This is now fixed.

CAMS Radiation service uses a model named Apollo from the DLR to compute clouds. Apollo cloud data were not properly processed on 2015-05-07, 2015-09-07 and 2015-09-08.
This is now fixed.

#642

This bug occurs with CAMS Radiation verbose mode.

A bug occurs in the latest version (11.0.96) of Internet Explorer on Windows 7: OpenStreetMap world map is not displayed properly.

This problem is fixed by displaying an alternate map for IE 11: OpenStreetMap Hot.

www.soda-pro.com is a mirror of www.soda-pro.com.

We fixed a bug: McClear and CAMS Radiation WPS did not work on www.soda-pro.com when www.soda-pro.com is down.

A classic error when using CAMS Radiation and McClear WPS is to forget to encode the "@" character.

We detect now this error and return a proper error message.

Since 2018-06-20, CAMS Radiation Service notifies gaps in Apollo cloud data by:

• large gaps (1 full day or more) are not filled and are notified with irradiation=NaN
• gaps smaller than 1 day are filled and are notified with reliability=0

A side effect was that monthly irradiation are equal to NaN as soon as one day of Apollo data is missing.

As our users notified us that this behavior is not user-friendly, we restored the gap filling for monthly requests.

#635

Cloud coverage value in CAMS Radiation Service can be unspecified

In verbose mode, CAMS Radiation service outputs a Cloud coverage value.

It can be unspecified. Up to now, in such case CAMS Radiation Service returned 0. Now, "unspecified value" is -1.

So possible values for cloud coverage are :

• [0; 100] : for valid value.
• -1 : if not specified,

We fixed a mathematical bug in the (latitude, longitude) position of the pixels of the Meteosat satellite images.

This fix changes (a little) the result of all CAMS Radiation requests.

CAMS Radiation Service version number is increased to 3.2.

CAMS Radiation service uses a model named Apollo from the DLR to compute clouds. Since 2018-06-01, the DLR is experiencing problems to retrieve the Meteosat satellite images on time.

UPDATE: this problem is fixed since 2018-06-21.

So far, CAMS Radiation Service sometimes notified a gap with irradiation=NaN and sometimes with reliability=0.

This new version is more coherent:

• large gaps (1 full day or more) are not filled and are notified with irradiation=NaN
• gaps smaller than 1 day are filled and are notified with reliability=0

McClear, CAMS Radiation and Helioclim-3 version 5 use aerosols, ozone and water vapor computed by the C-IFS meteorological model from ECMWF.

This model will be updated on 2018-06-26.

We fixed a few bugs in McClear clear sky model. The results are slightly modified in October 2012 and September-November 2015.
We increased McClear version number to 3.1.

This change in McClear modifies also CAMS Radiation and Helioclim-3 models.

CAMS Radiation service uses a model named Apollo from the DLR to compute clouds. On 2017-10-21, a fire occured in the DLR server room.

From this date to 2018-02-23, the DLR has ben able able to provide us cloud data with a delay (typically up to d-3 instead of d-2).

This problem is now fixed.

#566 Increase CAMS Radiation limit to 40 requests per day

From 2018-01-10 to 2018-01-17, the web site http://solar.atmosphere.copernicus.eu was not available.

http://solar.atmosphere.copernicus.eu is a subset of www.soda-pro.com with only CAMS Radiation and McClear services.

Fixed bug #555 "Since 2017-11-22 update, CAMS and MERRA forms are not displayed in Chrome"

CAMS Radiation service uses a model named Apollo from the DLR to compute clouds. On 2017-10-21, a fire occured in the DLR server room. Since this date, the DLR is able to provide us cloud data with a delay (typically up to d-3 instead of d-2).

This problem is not yet fixed.

The handling of monthly requests was not uniform when start and end dates do not define entire months.
We modified Helioclim-3, Helioclim-1, MERRA-2 and GFS to enlarge partial months requests to entire months. For request overlapping current month, McClear and CAMS Radiation gave "nan" values. So for coherency purpose, the monthly requests are first cropped to full month, and the request is then run with those cropped months.

CAMS Radiation Service features a new version 3 with:

• McClear clear sky model v3  (see #494 below)
• An empirical bias correction algorithm aiming at reducing errors in irradiation and clearness index. It has been created by fitting functions onto measurements performed at several BSRN stations taken globally.

NB: By construction, the CAMS Radiation Service performs the calculation of the radiation on-the-fly at the request of any user. It processes on-the-fly the necessary information that is stored in CAMS and does not create a proper database of the results. Hence, the new version 3 applies from now on back to 2004-02-01.

Access:

• the CAMS Radiation version 3 service
  
• the new CAMS Radiation validation results
  
• The bias correction description
  
• The description of the new CAMS McClear model
  

No impact.

If possible, users should redo all past CAMS Radiation Service requests to enjoy the improvements of v3.

McClear clear sky model features a new version 3 that fixes discontinuities in space and time (during day as well as seasons), uses less memory and reduces (a little) the error when compared to ground measurements.

Access:

• the McClear version 3 service
  
• the McClear version 3 documentation
  
• the new HC3v4 and v5 validation results over the BSRN stations
  

Can potentially impact HelioClim-3 version 5 by 1 to 2%.

McClear, CAMS Radiation and HelioClim-3 version 5 services use a meteorological model from ECMWF to describe the atmosphere.
Since 2017-09-26, this model has been upgraded to version 43r3.

McClear, CAMS Radiation and Helioclim-3 version 5 services use a meteorological model from ECMWF to describe the atmosphere.
This model is updated daily. This update has been broken from 2017-08-27 to 2017-08-30.

#434

Increment the max number of CAMS Radiation requests per day

Increment the max number of CAMS Radiation requests per day from 15 to 25.

See http://www.soda-pro.com/web-services/radiation/cams-radiation-service

CAMS Radiation users are allowed to run up to 25 requests per day :-)

Create solar.atmosphere.copernicus.eu microsite

Create solar.atmosphere.copernicus.eu microsite with 2 pages:

http://solar.atmosphere.copernicus.eu/cams-radiation-service

http://solar.atmosphere.copernicus.eu/cams-mcclear.

No impact

No impact

#421

Upgrade of the CAMS (Copernicus Atmosphere Monitoring Service) real-time data assimilation and forecasting system on 2017-01-24

The CAMS numeric weather model that provides inputs to McClear, CAMS Radiation and Helioclim-3 version 5 irradiation models has been improved on 2017-01-24

No impact

#416

#425

Bug fixes in CAMS Radiation service monitoring

Bug fixes in CAMS Radiation service monitoring

Restricted to CAMS project members

The McClear and CAMS Radiation (ex MACC-RAD) web services have been officially moved from toolbox.webservice-energy.org to www.soda-is.com on 2016-04-20.

We kept the old web services on toolbox.webservice-energy.org during a transition period. They have been removed today.

Users of the obsolete web services on toolbox.webservice-energy.org must upgrade their code.

See http://www.soda-pro.com/help/cams-services/cams-radiation-service/automatic-access

#191

Bug fixes in CAMS Radiation service monitoring when accessed from DLR

Bug fixes in CAMS Radiation service monitoring when accessed from DLR

Restricted to CAMS project members

#397

CAMS radiation gives different results on www.soda-is.com and www.soda-is.com since 2016-10-05

CAMS radiation gives different results on www.soda-is.com and www.soda-is.com since 2016-10-05

If possible, users should redo CAMS radiation requests computed by www.soda-is.com and www.soda-is.com from 2016-10-05 to 2016-10-18

#390

Fixed McClear atmospheric profiles

Fixed weights > 1.0 and < 0.0 in McClear atmospheric profiles

No impact

#191

CAMS Radiation service monitoring works on www.soda-pro.com

CAMS Radiation service monitoring works on www.soda-pro.com

Restricted to CAMS project members

-

Changes in McClear and CAMS Radiation web pages

Added hyperlinks to https://atmosphere.copernicus.eu/user-support-0 (now broken, March 2019) in McClear and CAMS Radiation web pages

No impact

#381

Download errors after CAMS system upgrade on 2016-06-21

After CAMS system upgrade on 2016-06-21, we encountered download errors up to 2016-08-12.

No impact

#379

Fixed bug in Apollo Cloud monitoring

Fixed bug in Apollo Cloud monitoring

Restricted to CAMS project members

-

Upgrade of McClear WPS

Added verbose HDF5 output to McClear WPS

No impact

#369

Upgrade of the CAMS real-time data assimilation and forecasting system on 2016-06-21

The CAMS numeric weather model that provides inputs to McClear, CAMS Radiation and Helioclim-3 version 5 irradiation models has been improved on 2016-06-21. The new model uses a 0.4 degree grid (about 40 km) and computes now hourly previsions twice a day.

No impact

-

McClear and CAMS Radiation web pages have a new look & feel

http://www.soda-pro.com/web-services/radiation/cams-radiation-service and http://www.soda-pro.com/web-services/radiation/cams-mcclear use now the look & feel of the Copernicus Atmosphere Monitoring Service (http://atmosphere.copernicus.eu) that funded these services.

No impact

-

Change in McClear and CAMS Radiation requests header

Added link to release notes information on  www.soda-pro.com in McClear and CAMS Radiation .csv output files

No impact

#319

Increased CAMS Radiation Service limit

Increased CAMS Radiation Service limit from 5 to 15 requests per day

No impact

#319

McClear and CAMS Radiation WPS are moved to www.soda-is.com

McClear and CAMS Radiation WPS are moved from test server http://toolbox.webservice-energy.org to production servers www.soda-is.com and http://www.soda-is.com

McClear and CAMS Radiation WPS users should update their code.

#319

MACC-RAD service is renamed CAMS Radiation

The MACC-III project that funded MACC-RAD and McClear services is replaced by the Copernicus Atmosphere Monitoring Service (http://atmosphere.copernicus.eu). A consequence is that the MACC-RAD service is renamed CAMS Radiation.

The get_macc_rad WPS is renamed get_cams_radiation. Users should update their code.

#268

Fixed bug in McClear web form

Bug fix: sometimes, McClear web form http://www.soda-pro.com/web-services/radiation/cams-mcclear fails with a "Connection reset" error

No impact