Recent Posts

Pages: [1] 2 3 ... 10
1
Data and Software Questions / Re: CPF Download from new https server
« Last post by Toshimichi Otsubo on February 06, 2020, 11:08:14 AM »
Hi Daniel and all,

I tried a different way - their "secondary" option ftp-ssl seemed easier in our environment.

Here is how I do using 'lftp' (command-line ftp client):

$ lftp -f cddis-cpf.lftp

where cddis-cpf.lftp (or any file name) is a text file like:
---
Code: [Select]
open -d -u anonymous,<YOUR EMAIL ADDRESS> -e 'set ftp:ssl-force true' gdc.cddis.eosdis.nasa.gov
set xfer:clobber true
cd /pub/slr/cpf_predicts/current
lcd <YOUR LOCAL DIRECTORY>
mirror -e -x MD5SUMS -x SHA512SUMS
exit
---

(Replace the two "< >" parts.)

Then, the local directory will contain the same files as the "/pub/slr/cpf_predicts/current" directory of CDDIS.  The lftp mirror downloads only newer files, so it will be quicker for the second time.

Toshi
2
Data and Software Questions / Software: CPF to Range+Az+El?
« Last post by Toshimichi Otsubo on February 01, 2020, 05:54:35 AM »
Are there any publicly available software that can produce the [Range, Az, El] time series from CPF for a given station position?  Toshi
3
Data and Software Questions / CPF Download from new https server
« Last post by danielhampf on January 16, 2020, 09:28:24 AM »
Hello all,
you've probably seen the emails about the NASA CPF server changing to https. I now adapted our python script in order to download the CPFs from that new server. I used the requests package rather than curl as I suggested by CDDIS examples. It's a bit tricky since they decided to use a login system which is not very suitable for automated download. Anyway, it works now. I post the code here for your convenience.
A few notes to the code. It's made to work with python3 and the python requests package installed. You need to obtain Earth Data login credentials first and then insert them here in the code (line ~37 / 38). The code will download the V1 CPFs, if you want the new ones, you have to change the target folder. Please note that the script will delete all existing files in the local target folder. If you use the function in a graphical interface, your callback function can update a progress bar etc. If it returns "false", the downloads will be cancelled. The function will return the number of downloaded CPFs.
The script is not entirely by me, it also uses code I found on the Earth Data site.
Comments and cheers welcome, as always.
Daniel




import os
from glob import glob
import requests

class SessionWithHeaderRedirection(requests.Session):
    AUTH_HOST = 'urs.earthdata.nasa.gov'
    def __init__(self, username, password):
        super().__init__()
        self.auth = (username, password)   
   
   # Overrides from the library to keep headers when redirected to or from
   # the NASA auth host.
    def rebuild_auth(self, prepared_request, response):
        headers = prepared_request.headers
        url = prepared_request.url
 
        if 'Authorization' in headers:
            original_parsed = requests.utils.urlparse(response.request.url)
            redirect_parsed = requests.utils.urlparse(url)
 
            if (original_parsed.hostname != redirect_parsed.hostname) and \
                    redirect_parsed.hostname != self.AUTH_HOST and \
                    original_parsed.hostname != self.AUTH_HOST:
                del headers['Authorization']
        return   
         
def download_CPFs_ssl(local_dir, update_callback):
    # remove old files
    if not os.path.isdir(local_dir):
        os.mkdir(local_dir)
    filelist = glob(os.path.join(local_dir, "*"))
    for f in filelist:
        os.remove(f)

    # define urls and credentials
    url = "https://cddis.nasa.gov/archive/slr/cpf_predicts/current/"
    username = ""
    password = ""

    # make the request to the web site to get filenames
    session = SessionWithHeaderRedirection(username, password)       
    response = session.get(url + "*?list")
   
    # check if response is okay
    if response.status_code is not requests.codes.ok:
        log.error("Could not connect to CPF server. HTML code: %d" % (response.status_code))
        return False   
   
    # parse the response and make list of filenames
    lines = response.text.split('\n')
    filenames = []
    for line in lines:
        if line.startswith("#"):    # comment lines
            continue
        if line.strip() == "":      # empty lines
            continue
        filename, size = line.split()
        filenames.append(filename)       

    # download each file and save it
    excl_list = ["MD5SUMS", "SHA512SUMS", "index.html"]
    for i, filename in enumerate(filenames):
        if filename in excl_list:
            continue
        filepath = url + filename
        response = session.post(filepath)
        with open(os.path.join(local_dir, filename), "wb") as f_out:
            f_out.write(response.content)
        keep_running = update_callback(100. * i / len(filenames))
        if not keep_running:
            break
    return i
       
       
       
if __name__ == "__main__":           
    def print_progress(p):
        print(p)
        return True
               
    download_CPFs_ssl("./CPF/", print_progress)
           
4
Station Equipment Questions / Re: Infrared thermal usb camera
« Last post by ewan.schafer on December 18, 2019, 01:00:51 PM »
Hi Matt,

Just want to echo what Johann said: the sensor seems nice but make sure you're getting one with a decent sized aperture to make sure you have enough sensitivity. Our camera can struggle distant aeroplanes when they're lower in the sky & under certain weather conditions.

This one seems like a nice model:
https://www.flir.co.uk/products/boson/?model=20640A006

But unfortunately, because they probably have to grow some kind of fancy crystal for the lens, I expect that the price is going to scale significantly with the amount of glass that you're buying.

Let me know how how it goes if you decide to get one, they could be interesting for us too!
5
Station Equipment Questions / Re: Infrared thermal usb camera
« Last post by Johann on December 06, 2019, 01:56:37 PM »
Hi Matt,

as you may already know, we bought our camera from InfraTec. I think the camera itself has almost the same specifications. However to increase the sensitifity we chose the greatest entrance lens that was available. In our case the lens diameter is 120 mm, while the field of view is 8 x 6 degree. I would not recommend to increase the field of view, because you are loosing sensitifity and resolution. We would prefer to have an even narrower field of view to reduce false alarm. But that is probably also depending on the applied object detection algorithm. Unfortunately i only know the price for the camera including the lens. It starts from ~29k, with some extras like low-noise-detector and increased resolution (1024x768) we ended up at almost 40k.
The cameras that you found look promising. With 60 Hz they provide an even higher frame rate. It would be nice to hear the price.

Best regards,
Johann
6
Station Equipment Questions / Infrared thermal usb camera
« Last post by Matt Wilkinson on November 05, 2019, 11:36:55 AM »
Hi
At the recent Technical Workshop in Stuttgart, we heard from a number of people operating infrared cameras.  The reports were encouraging but the cameras are expensive. Could i ask what people on here think of these cheaper cameras?:

https://www.flir.co.uk/products/boson/?model=20640A012
    Longwave infrared; 7.5 µm – 13.5 µm
    640 x 512
    60Hz
    12 µm pixel

Thanks!
7
Data and Software Questions / Re: CRDv2 experience (as a provider)
« Last post by Matt Wilkinson on October 07, 2019, 01:20:08 PM »
Please also see this earlier post on CRDv2.
http://sgf.rgo.ac.uk/forumNESC/index.php?topic=51.0

I could not recover the author's name.  Was it you??
8
Data and Software Questions / CRDv2 experience (as a provider)
« Last post by Matt Wilkinson on October 07, 2019, 01:18:10 PM »
How are you getting on with generating version 2 CRD SLR data files?

At Herstmonceux, we are about to start submitting our data in CRDv2 format. I wrote a Python script to generate the files following a similar method to how we write the CRDv1 files in FORTRAN.

It wasn't too much trouble and i think it is all correct now. If you have not looked at doing this yet, there are changes to be aware of in some of the fields and there are also additional fields:
  • H5 Prediction Header
  • C5 Software Configuration
  • C6 Meteorological Instrumentation Configuration
  • C7 Calibration Target Configuration *new*
  • 41 Calibration Detail Record
  • 42 Calibration "Shot" Record *new*

One major change is the recording of the calibration records.  In the v2 version only one '40' record should be included with additional calibrations recorded as '41'.  The epoch of the '40' record should be the at the middle of the pass segment and the calibration value determined from pre and post measurements.

See the latest CRDv2 document from the ILRS website: https://ilrs.cddis.eosdis.nasa.gov/data_and_products/formats/crd.html

It was clear to me in doing this that more data can be included in the CRD files than we are providing at Herstmonceux.  In time, i will look at including the '30' records for Pointing angles.

We also don't collect the information requested in the '21' Meteorological Supplement Record. Is anyone collecting this information?:
  • Wind speed (m/s)
  • Wind direction (degrees azimuth, North is zero)
  • Weather conditions (two-digit SYNOP/WMO “present weather” code, or “rain”, “snow”, “fog”, “mist”, “clear”, “na”, etc.)
  • Visibility (km)
  • Sky clarity (i.e., zenith extinction coefficient)
  • Atmospheric seeing (arcsec)
  • Cloud cover (%)
  • Sky temperature in degrees Kelvin
We do however collect visibility values and so i will look in to including this in the future. Is there an instrument that anyone is using that reliably provides the rest of this information?

Please share your experience in providing CRDv2 below. Any questions or advice welcome

Thanks
9
ILRS Stations / A little article with Yarragadee in the background
« Last post by Yarragadee on July 11, 2019, 07:04:04 AM »
And some nice shots of the laser and the VLBI dish.

https://westtravelclub.com.au/stories/shooting-for-the-stars

Cheers, Sandy
10
Mission Tracking Feedback / Recovered: Radioastron
« Last post by Matt Wilkinson on June 14, 2019, 02:18:14 PM »

Lavochkin Association did not manage to establish communication with the Spectr-R satellite. Attempts continued from 10 January to 30 May 2019. The State Commission examined the satellite’s technical condition on 30 May 2019 and decided to finish the RadioAstron observing program. The satellite successfully operated for 7.5 years instead of the originally planned 3 years. The link with the satellite was lost due to the very long exposure to the space radiation which has affected the onboard low-gain antenna communication system. Currently, the Astro Space Center is completing the data transfer, correlation and archiving of the vast amount of unique scientific data. International science teams continue to process, analyze and publish the results.
Pages: [1] 2 3 ... 10