Title Goes Here

Title Goes Here

Working Group on Calibration and Validation (WGCV): 43 Infrared Visible and Optical Sensors (IVOS) subgroup: report Nigel Fox NPL (with UKSA support) WGCV 43 Working Group on Calibration and Validation Summary of activities IVOS 30 @ ESTEC,Noordwijk, Holland. hosted by ESA Mar 2018 24 agency/orgs represented 30 attendees + 7 remote Most themes and topics (work-plan discussed or summarised Special Projects: 4 th MTF workshop (Mar 2018) RadCalNet team met Mar 2018 Various telecons

~ PICSCAR webex (Joint GSICS) 2nd Meeting (4th inc pre-cursors) Surface Temp Val workshop (Oct 17) O-colour vicarious Cal comparisons started April 2017 (continues) Rad cal workshop (Europe focus) Aug 17) Terminology task team established mett Working Group on Calibration and Terms of Reference 1.

Promote international and national collaboration in the calibration and validation of all IVOS member sensors. 2. Address all sensors (ground based, airborne, and satellite) for which there is a direct link to the calibration and validation of satellite sensors; 3. Identify and agree on calibration and validation requirements and standard specifications for IVOS members; 4. Identify test sites and encourage continuing observations and inter-comparison of data from these sites; 5. Encourage the preservation, unencumbered and timely release of data relating to calibration and validation activities including details of pre-launch

and in flight parameters. 6. In the context of calibration and validation encourage the full consideration of traceability in all activities involved in the end-to-end development of an EO product including appropriate models and algorithms. Working Group on Calibration and IVOS: Vision To facilitate the provision of fit for purpose information through enabling data interoperability and performance assessment through an operational CEOS coordinated & internationally harmonised Cal/Val infrastructure consistent with QA4EO principles. Pre-flight characterisation & calibration Test sites Comparisons Agreed methodologies Community Good Practises

Interchangeable/readable formats Results/metadata - databases Key Infrastructure to be established and maintained independent of sensor specific projects and/or agencies Working Group on Calibration and Work plan Structured into themes and led by champions (effectively vice chairs for CEOS WGCV constitution) (Plus specific projects) Look to develop good practises Organise comparisons Shared learning (research activities) Shared infrastructure / tools / Methods surface Recommendations as needed Land

reflectance - Czapler Myers (U of Arizona USA) Ocean colour (link to IOCCG, VC-OCR etc) - Zibordi (JRC, EU) & Murakami (JAXA JPN) Surface Temperature (link to VC-SST, GHRSST) - Corlett (U of Leicester, UK) Geo spatial image quality - Helder (SDSU, USA) & Viallefont (ONERA F) Atmospheric Correction (Link to AC subgroup) - Thome (NASA, USA) RT codes (context of IVOS use in calibration) -? Working Group on Calibration and

Specific projects/cross-cutting RadCalNet PICSCAR (with GSICS) - Henry (CNES, F) Lunar (led by GSICS) - Wagener (Eumetsat) SST/LST cross-comparison (+ VC-SST & LPV - Fox (NPL, UK) (instrument Cal for LST) O-Colour Vicarious Cal comparisons UK)

Others in progress/development/related - Bouvet (ESA) - Fox (NPL, Establishing a CEOS Reference and method of use for L1 radiometric interoperability (with GSICS) (including potential tools/databases) Good practise for convolving spectral data sets (solar/surface/sensor bandwidth) Selection of Reference Solar irradiance spectrum(CEOS WGCV (sub-groups) & GSICS) Working Group on Calibration and IVOS 30 Discussion Topics

Summary of workshops/task groups, MTF, RadCalNet, PICSCAR Terminology Sensor pre-flight calibration workshop Sensor Pre- and In- flight Cal and Uc assessment Inc Moon, Stars New Sensors Surface reflectance (level 2) validation Collaborations/interactions WGCV, GSICS, VCs, Climate, Carbon . Metrology and Uc evaluation. Working Group on Calibration and MTF activities & comparison

Task group created in 2015 and very active and successful providing a valuable resource to the small but increasing community (particularly commercial sensors) Working Group on Calibration and Establish good practise and community references Catalogue is being migrated to CalVal Portal Following completion of list of sites propose to CEOS WGCV acceptance panel for formal CEOS Label to aid community & interoperability Working42 Group on Calibration and RECOMMENDATION agreed at WGCV Comparison of methods Deviation to the model StdSystem_1 (2017) ACT MTF difference 0,1 G

0,05 F E 0 D 0 0,1 0,2 0,3 0,4 0,5 0,6

0,7 -0,05 0,8 0,9 1 C B A -0,1 Normalized frequency - Results, methods drafted into paper for submission to Optics letters in May/June 18 referencing CEOS data set and exercise (essentially a reference guide) - Create reference data set and access policy to allow blind-testing from Cal/Val Portal as CEOS test set agreed at CEOS WGCV 42 Next steps : List of metrics besides MTF curve: RER, RER slope, FWHM,

Establish a list of relevant information or feature for each metric: for instance, link with the requirements, sensitivity, link with other metrics, potential use Working Group on Calibration andset Evaluate some metrics and compute them as extensions to test data RadCalNet Working Group on Calibration and Documentation Working Group on Calibration and Request to CEOS WGCV R.2017-1

Following the successful Beta Testing of RadCalNet, IVOS is looking for CEOS-WGCV endorsement for opening as an operational network (once the remaining tasks listed in the RadCalNet WG minutes are complete). IVOS asks WGCV to set up the necessary governance structure to approve RadCalNet sites. Following full operational access: IVOS requests agencies/site owners to consider enhancing RadCalNet through contributing new sites RadCalNet WG has now peer reviewed the Uc of current sites and assessed their suitability and request that WGCV accept And open to public Working Group on Calibration and PICSCAR Joint with GSICS lead P Henry CNES Pseuodo invariant sites used for Rad L1 Cal/comparison/monitoring Work-plan Bi-monthly community Telecons/webex to discuss methods and how to establish/compare (BRDF, Atmosphere, surface spectral reflectance) Organise comparison exercises to assess community state of

the art CNES to establish draft web portal Hosted on/via CalVal portal Good practises on how to use Repository of observations Repository of comparison results Characteristics of PICS and tools for users NPL & CNES to consider how to evaluate Uc Goal to establish a TOA Radiance simulator Working Group on Calibration and Comparisons

Evaluate consistency of data extraction/data sets for Landsat 8 and Sentinel 2 (Libya 4) (initially some variances) Define and supply a common set of data to participants for independent processing Exercise 1: Intercompare L8 against S2A as Reference Exercise 2: Intercompare MODIS Aqua against MERIS Exercise 3: Assess the BRDF of the site (own model) to normalise TOA reflectance Sensor MERIS S2A/MSI L8/OLI MERISto a specific

MODIS/Aqua geometry. Green Band B3 (560) B3 (562) B5 (560) B4 (555) Red Band B4 (665) B4 (655) B7 (665) B1 (645) NIR band

B8A (865) B5 (865) B13 (865) B2 (858) Five teams so far provided results for Exercise 1) Working Group on Calibration and PICSCAR portal to be linked from Cal/Val portal Working Group on Calibration and Draft content Working Group on Calibration and Project to evaluate possible new PICS and surface reflectance

Revisit selection of possible PICS sites Establish database of measured characteristics of PICS sand Attempt to build a BRF model (s) Working Group on Calibration and FRM4STS: Fiducial Reference measurements for validation of Surface Temperature from Satellites: Results of Lab and near lab comparisons: A contribution to SST-VC and LPV All info is available here: www.frm4sts.org Results submitted as 4 papers Working Group on Calibration and Validation International workshop Oct 2017

~ 40 attendees from 4 continents representing sat surface T validation community. Endorsed good practises for Land, Ocean and Ice Land already incorporated into LPV good practice guide 20 Some Key Recommendations FRMs should be encouraged need more sites, more match-ups and more comparisons Super-sites with WMO? particularly over land (also urban, mountains. Polar ) Research to look at scaling point to satellite, heterogeneity, global representativeness

Research to look at effects of T skin to depth water, snow, Ice Training / Case studies on Uc estimation and analysis + good practice guides on measurements and instruments Comparisons designed to account for operational conditions (low/high ambient T) Ship based multi laterals for oceans Cloud detection/masking (day/night) Satellite and Validation 21 Some Key Recommendations

Link Satellites to Validation compare traceability and reference standards (not rely on models) Compare retrieval algorithms (using standardised data) More (traceable Buoys) consider triple sensors for redundancy, recoverability? Look for synergy in other observations e.g. passive microwave and IR could be encouraged need more sites, more match-ups and more comparisons Super-sites with WMO? particularly over land (also urban, mountains. Polar ) 22 Level 2 Surface Reflectance

Product Validation Thoughts CEOS WGCV 43 Prepared By: Cody Anderson*, Dennis Helder*, and Jeff Czapla-Myers** *USGS EROS, **University of Arizona On behalf of IVOS to propose initiation of a new activity ultimately cross sub-group at WGCV level but initially some scoping by IVOS of underpinning radiometry Working Group on Calibration and Surface Reflectance Validation Thoughts (1) A global project/opportunity Many sensors Many land surfaces Many atmospheres No one organization can exhaustively cover these! Initial Activities Geoscience Australia/CSIRO field campaigns

ESA Hypernets USGS/NASA only initial discussions Where is the CEOS home for this: LPV or IVOS? How do we begin? One possible approach Identify interested organizations Define acceptable approaches Develop standard/best practices Mechanism for identifying and accepting high quality data Develop mechanism for sharing data Develop sustainability plan. 24 Surface Reflectance Validation Thoughts (2) Develop instrumentation to appropriately monitor sites Field campaigns Automated instrumentation UAV Partner with other organizations producing similar high quality surface reflectance data sets Ensure traceability of measurements

Measurement uncertainty required Use stable PICS to monitor long term trends Use well known Sahara sites Consider using globally distributed sites Range of surface reflectance values Range of atmospheric parameters Understand uncertainties 25 Surface Reflectance Validation Thoughts (3) An approach based on Landsat Surface Reflectance Product Validation Use only sites/targets with surface reflectance directly measured Develop standard for acceptable surface reflectance measurement approach Show traceability to National Standards Develop uncertainty model Identify sites that span a broad range of atmospheric types Tropical Atmosphere Mid latitude Summer Mid latitude Winter Subarctic Summer Subarctic Winter Standard US Atmosphere

Identify sites that span a broad range of land (water?) cover types Agriculturecrops/hay Forestdeciduous/coniferous Developed (cities) Shrubland Grassland Wetland Coastal Open Water Snow/Ice 26 Surface Reflectance Validation CEOS WGCV, 1013 Apr 2018 Surface reflectance is determined during vicarious calibration Can be used to validate higher-order data products In situ surface reflectance measurement techniques Typical technique: reference panel and spectroradiometer

Panel calibration should be linked to metrology standard for SI traceability Current uncertainty estimate: ~2% Automated systems (e.g. RadCaTS/RadCalNet) Use different techniques to measure surface reflectance Relies on solar model to determine surface reflectance Current uncertainty estimate: ~3.55.0% (U of AZ RadCaTS) 27 2nd GSICS/CEOS workshop on moon Xian China, Nov 2017 2014 14 agencies 26 attendees 2017 22 agencies 60 attendees USGS ROLO new funding Starting to look at new uses of moon e.g MTF (link with IVOS) 28 ESA project to measure lunar

irradiance as a Cal reference Define a strategy to derive the model regression coefficients (ROLO based) from the lunar measurements Derive regression coefficients from database of measurements Measurements uncertainty propagation in to the model parameters / regression coefficients 29 Terminology Uncertainty is not the same as Error

Working Group on Calibration and Task group on vocabulary formed Name Institute Name Institute Emma Woolliams NPL Stefanie Holzwarth DLR Steffen Dransfeld

ESA-ESRIN Cindy Ong CSIRO Lingling Ma AOE-CAS Carol Bruegge JPL-NASA Xinhong Wang AOE-CAS Hirokazu Yamamoto Cibele Teixeira Pinto

South Dakota State University AIST DLR Paolo Castracane ESA-ESRIN Martin Bachmann Select some key terms and also concepts create a dictionary / thesaurus and also small video explanations where useful Working Group on Calibration and Ocean Colour FRM comparisons for OCR-VC and IOCCG To help address IOCCG white paper Run comparisons of validation instruments

Lab Ocean Ref standards Ensure SI traceability and Uc to SI Draft protocols for how to establish/maintain traceability Review requirements for future infrastructure ESA sponsored project www.FRM4SOC.org To support CEOS VC-OCR First two phases of comparisons completed 11 participants from across the globe Working Group on Calibration and Benchmark Calibration/climate sensors Plan to have a CEOS/GSICS workshop in Europe in 2019 - CLARREO - TRUTHS - Chinese Benchmark sensor Working Group on Calibration and

A new community 3D RT code [email protected] Working Group on Calibration and Summary IVOS active team expanding (good global coverage- agency and industry) Thematic projects working effectively with motivated champions: Number sometimes make logistics for meetings an issue but are working well between IVOS plenaries via webex etc Recommendations proposed and agreed at CEOS WGCV 42 (MTF sites and comparisons etc are being implemented.

Looking for good examples of impact Many collaborations with GSICS and supporting VCs Keen to revitalise and use Cal/Val portal as the community interface Plan to have session on Hyperspectral imagers at next IVOS. Working Groupsome on Calibration and Still need to engage and bring

on-board agencies

Recently Viewed Presentations

  • Esercitazione - University of Cagliari

    Esercitazione - University of Cagliari

    Title: Esercitazione Author: aa aa Last modified by: Anna Altieri Created Date: 2/2/2013 3:55:14 PM Document presentation format: Presentazione su schermo (4:3)
  • William Stallings, Cryptography and Network Security 3/e

    William Stallings, Cryptography and Network Security 3/e

    Cryptography and Network Security Third Edition by William Stallings Lecture slides by Lawrie Brown Basic Terminology plaintext - the original message ciphertext - the coded message cipher - algorithm for transforming plaintext/ciphertext key - info used in cipher known only...
  • Chapter 19 Petits Fours On Baking Labensky et

    Chapter 19 Petits Fours On Baking Labensky et

    Petits Fours - Miniature Pastries Guidelines are: No more than one or two small bites Represent a variety of textures and flavors Be visually attractive Complement whatever foods precede or accompany them without duplicating their flavors Petits Four Varieties Petit...
  • The Ocean Basins The Water Planet  Nearly 3/4ths
  • The Cold War - Henry County School District

    The Cold War - Henry County School District

    1950 Red Scare. McCarthy made a public accusation that more than two hundred "card-carrying" communists had infiltrated the United States government. Though eventually his accusations were proven to be untrue, and he was censured by the Senate for unbecoming conduct....
  • Willingness to Pay for Hermetic Grain Storage Bags

    Willingness to Pay for Hermetic Grain Storage Bags

    A survey prior to the auction elicited farmer characteristics, including household wealth (in quintiles of an asset index), amount of grain stored (in kg), knowledge about aflatoxins and how bags work (0-12 questions answered correctly), and whether they had attended...
  • ME-330 Introduction - engr.usask.ca

    ME-330 Introduction - engr.usask.ca

    ME 330 Introduction. Revision. For the final exam. Contact: [email protected] I prefer to meet people during office hours. Email me if you need to meet me at another time.
  • The Federal Geospatial Platform CEOS WGISS September 2015

    The Federal Geospatial Platform CEOS WGISS September 2015

    A three-year Government of Canada (GC) Initiative taking place between 2014 and 2017 that: Supports horizontal efforts to advance economic growth, social cohesion, well-being, and environmental management. Will fundamentally change the way the GC shares, uses, and manages geospatial assets...