SystemError in aplib.tsc_time on FreeBSD/EC2

[bobveznat]

import coro
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.6/site-packages/coro/init.py", line 131, in
from coro._coro import *
File "aplib.tsc_time.pxd", line 28, in init coro._coro (coro/_coro.c:49571)
File "aplib.tsc_time.pyx", line 275, in init aplib.tsc_time (aplib/aplib.tsc_time.c:11561)
File "aplib.tsc_time.pyx", line 254, in aplib.tsc_time.get_ticks_per_sec (aplib/aplib.tsc_time.c:1045)
SystemError

uname -a

FreeBSD ip-10-170-30-5 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Aug 4 15:39:43 UTC 2011 root@ip-10-17-5-252:/usr/obj/i386/usr/src/sys/XEN i386

This is a t1.micro instance.

sysctl machdep

machdep.rtc_save_period: 1800
machdep.adjkerntz: 0
machdep.wall_cmos_clock: 0
machdep.disable_rtc_set: 0
machdep.disable_mtrrs: 0
machdep.guessed_bootdev: 0
machdep.idle: hlt
machdep.idle_available: spin, mwait, mwait_hlt, hlt, acpi,
machdep.prot_fault_translation: 0
machdep.panic_on_nmi: 1
machdep.kdb_on_nmi: 1
machdep.xen_disable_rtc_set: 0
machdep.independent_wallclock: 1

sysctl -a | grep tsc

kern.timecounter.smp_tsc: 0
kern.timecounter.invariant_tsc: 1

[markpeek]

Interesting...over the weekend I was looking into possibly using clock_getres()/clock_gettime() instead of TSC. In other words, create a plugable timer interface into shrapnel instead of just TSC. I need to do timings to see what the overhead would be for clock_gettime() but it might be a better choice for Linux and now the t1.micro instance. The TSC in shrapnel doesn't do a cpuid instruction to flush the instruction cache so it is probably the fastest. I've seen a posting showing timings of clock_gettime() faster than a cache flushed TSC. I'll just have to see how much of a performance change it would be...but having a clock is better than no clock.

[bobveznat]

I have this patch. Which works and appears reasonably accurate. But I don't really understand the ramifications of it. Eric is also reading FreeBSD source at this moment.

With this patch time.time() gets totally borked (Eric tells me that time.time() is monkey patched when you import coro).

bobveznat@2a8427f

[ehuss]

I am unable to determine exactly how or why Xen removes the machdep.tsc_freq sysctl. TSC is enabled and the cpu instruction works.

In a discussion with Mark and Bob about this. Regardless, we will need to have better timer support for other platforms. Using clock_gettime(CLOCK_REALTIME, ...) seems like the best option for Linux and BSD. Darwin will probably need to use mach_absolute_time. Having a design that can use different clock systems will be the best option. TSC has a lot of issues, too (calibrating it, for example).

BTW, I did some performance tests of clock_gettime. It's not horrible. On Bob's EC2 instance, I could call it about 1.25 million times per second. Still about 250 times slower than RDTSC, but probably won't be noticeable.

[samrushing]

How about CLOCK_MONOTONIC (or rather CLOCK_MONOTONIC_FAST)?

[markpeek]

I think CLOCK_MONOTONIC is the common choice for Linux and FreeBSD. But the OS's might need their own since the precision and how time shifts (time/ntp) appear to be different.