[SOLVED] RPI, GPS and python - fix time format problem

Moderators: adafruit_support_bill, adafruit

Forum rules
Talk about Adafruit Raspberry Pi® accessories! Please do not ask for Linux support, this is for Adafruit products only! For Raspberry Pi help please visit: http://www.raspberrypi.org/phpBB3/
Locked
User avatar
przemo
 
Posts: 43
Joined: Tue Jul 22, 2014 4:28 pm

[SOLVED] RPI, GPS and python - fix time format problem

Post by przemo »

I'm not sure uf that's the best place to post this problem - please move if there is a better location.

I have a rpi + adafruit gps board. Everything works fine with default gps settings. I'm using gps python module to handle gps parsing.
Log from the program:

Code: Select all

DEBUG:root:[GPS] timestamp: 1416150533.52, fix time: 2014-11-16T15:08:53.000Z, UTC: 2014-11-16T15:08:53.000Z, Satellites: 13, Used: 4
DEBUG:root:[GPS] timestamp: 1416150534.57, fix time: 1416150534.0, UTC: 2014-11-16T15:08:54.000Z, Satellites: 13, Used: 4
DEBUG:root:[GPS] timestamp: 1416150535.52, fix time: 2014-11-16T15:08:55.000Z, UTC: 2014-11-16T15:08:55.000Z, Satellites: 13, Used: 4
DEBUG:root:[GPS] timestamp: 1416150536.57, fix time: 1416150536.0, UTC: 2014-11-16T15:08:56.000Z, Satellites: 13, Used: 4
DEBUG:root:[GPS] timestamp: 1416150537.71, fix time: 1416150536.0, UTC: 2014-11-16T15:08:56.000Z, Satellites: 13, Used: 5
DEBUG:root:[GPS] timestamp: 1416150537.77, fix time: 2014-11-16T15:08:57.000Z, UTC: 2014-11-16T15:08:57.000Z, Satellites: 13, Used: 5
DEBUG:root:[GPS] timestamp: 1416150538.53, fix time: 1416150538.0, UTC: 2014-11-16T15:08:58.000Z, Satellites: 13, Used: 5
DEBUG:root:[GPS] timestamp: 1416150539.56, fix time: 2014-11-16T15:08:59.000Z, UTC: 2014-11-16T15:08:59.000Z, Satellites: 13, Used: 5
DEBUG:root:[GPS] timestamp: 1416150540.52, fix time: 1416150540.0, UTC: 2014-11-16T15:09:00.000Z, Satellites: 13, Used: 5

Important bits from the code:

Code: Select all

from gps import *

self.data = gps(mode=WATCH_ENABLE | WATCH_NEWSTYLE)
That part runs in a loop:

Code: Select all

self.data.next()
timestamp = time.time()
self.utc = self.data.utc
self.fix_time = self.data.fix.time
Code responsible for generating the log:

Code: Select all

self.occ.log.debug("[GPS] timestamp: {}, fix time: {}, UTC: {}, Satellites: {}, Used: {}".format(timestamp, self.fix_time, self.utc, self.satellites, self.satellites_used))
I cannot find out why fix time format keeps changing. It's like 2014-11-16T15:08:59.000Z or 1416150538.0
I can handle both, but I'd prefer to know what's going on. Anyone had the same problem?

P.S. I found this:

Code: Select all

                # Time can be either Unix time as a float or an ISO8601 string
                if type(self.fix.time) == type(0.0):
                    self.fix.time = self.utc
                else:
                    self.fix.time = isotime(self.utc.encode("ascii"))
in /usr/lib/python2.7/dist-packages/gps/gps.py but I don't understand why it's there and how to control in?
Last edited by przemo on Mon Nov 17, 2014 6:43 pm, edited 1 time in total.

User avatar
tdicola
 
Posts: 1074
Joined: Thu Oct 17, 2013 9:11 pm

Re: RPI, GPS and python - fix time format problem

Post by tdicola »

Hrm yeah I was looking a bit at the gps.py source (took a lot of searching but it's here: http://git.savannah.gnu.org/cgit/gpsd.g ... gps/gps.py ) and it's surprisingly complex. I don't really follow why it seems provide either an ISO string or float for time--perhaps the higher level gpsd daemon is providing the data? I don't really see anything in the gps.py code or your code that would make the time change format either way, so it likely is coming from gpsd. You might send them a message or check out their mailing list to see if it's a common thing other people have run into.

As an ugly fix, you could do something like that code shows and test the time value to see if its a float or string. That's what the check in the code you saw is doing:

Code: Select all

# Time can be either Unix time as a float or an ISO8601 string
if type(self.fix.time) == type(0.0):
    self.fix.time = self.utc
else:
    self.fix.time = isotime(self.utc.encode("ascii"))
The if is checking if the type of the fix.time member is the same as the type of the number 0.0 (a float). This is a pretty ugly check and I'm guessing was written long, long ago in an earlier version of python. Nowadays a more clear check would be something like:

Code: Select all

# Time can be either Unix time as a float or an ISO8601 string
if isinstance(self.fix.time, float):
    self.fix.time = self.utc
else:
    self.fix.time = isotime(self.utc.encode("ascii"))
As far as what to do in the code, if it's a float you could just keep the value as is and if it isn't a float you could use the strptime function to parse the date string into a time struct and pass that to mktime to get the floating point number. I don't know why the gps.py code and gpsd daemon doesn't just do this and instead has its clients deal with the different formats, but unfortunately it seems that's the way the code works. Here's a snippet to try somewhere in your code:

Code: Select all

# Convert string time value to float if necessary.
if not isinstance(self.fix.time, float):
    # self.data.fix.time is a string, so parse it to get the float time value.
    self.data.fix.time = time.mktime(time.strptime(self.data.fix.time, '%Y-%m-%dT%H:%M:%S.%fZ'))
You'll want to add an import for time at the top of the file if you use that snippet too.

User avatar
przemo
 
Posts: 43
Joined: Tue Jul 22, 2014 4:28 pm

Re: RPI, GPS and python - fix time format problem

Post by przemo »

Thanks for a very detailed reply! I wrote something like this:

Code: Select all

#fix_time is not always in seconds - make sure it is 
if type(self.fix_time) != type(0.0):
    ts = time.strptime(self.fix_time, "%Y-%m-%dT%H:%M:%S.000Z")
    self.fix_time = calendar.timegm(ts) 
but your version looks better.

Locked
Forum rules
Talk about Adafruit Raspberry Pi® accessories! Please do not ask for Linux support, this is for Adafruit products only! For Raspberry Pi help please visit: http://www.raspberrypi.org/phpBB3/

Return to “Adafruit Raspberry Pi® accessories”