java -Xmx4000m

If you don’t use this flag you will get an OutOfMemoryError exception as soon as your application exceeds the default heap size, which is typically set at only a few hundred megabytes.

However, I frequently found myself faced with a horrible performance experience when using a large heap. Java applications would freeze my entire (64-bit) Linux system for anywhere between 2 and 60 seconds! This happened regardless of the JVM used (I tried Oracle Java 1.7, Sun Java 1.6, GCJ 1.5). I verified that my system had plenty of memory available and was not swapping, hence a lack of memory was not to blame. A profiler revealed that these freezes were instead caused by an insane number of interrupts which ate 100% of all CPU cores in so-called “system” cycles.

The cause of these system freezes is Java’s garbage collection mechanism; a built-in automated memory management system which reclaims memory occupied by objects that are no longer in use. Whilst this feature makes programming in Java a bit easier than, say, C++; it comes with the disadvantage that garbage collection in a large heap can introduce a considerable overhead. Some collection algorithms deal less effectively with large heaps than others, and unfortunately in my case, Java appeared to be using a collection strategy which paused the application during the whole duration of each garbage collection run, hence resulting in frequent freezing.

The trick to avoid these freezes is to tell Java to use a collection strategy which runs concurrently to the application, hence avoiding lengthy interruptions of the entire process. This can be achieved using the “XX:+UseConcMarkSweepGC” flag, i.e.:

java -Xmx4000m -XX:+UseConcMarkSweepGC

There are in fact many more tuning parameters which can influence the behaviour of the garbage collection, but “UseConcMarkSweepGC” looks like the first obvious thing to try if you are experiencing annoying freezes in memory-intensive Java applications.

YouTube demographic statistics for the animation on comet PanSTARRS’s visibility.

Although the numbers are only based on the ~20% of viewers which were logged into a YouTube account while watching, statistics like these may reveal broad trends about the public interest in astronomy. If we were to assume that all people interested in astronomy are equally likely to have watched the animation, and if in addition we assume that these people are all equally likely to have a YouTube account regardless of their age/gender, then one might conclude that (middle-aged) men are twice more likely to seek for comet information than women. Interestingly, this is broadly consistent with the (unfortunate) trend of large male majorities in astronomy clubs and university departments.

There is no doubt that the above assumptions are wrong to some degree, and that the YouTube statistics are hence biased. It is not clear how severe the bias is however. I tried Googling for demographic statistics of YouTube users in general, but could not find consistent information. (Does anyone know a reliable source? Are 75% of YouTube users male anyway?!)

If the biases can be accounted for using a proper statistical analysis, then the analytics offered by science-themed YouTube videos would provide a way to measure the public interest as a function of age, gender and topic.

I plotted the visibility of Pan-STARRS in the video below. Green/yellow areas in the animation indicate parts of the world where the comet will be above the horizon (and the Sun at least six degrees below the horizon). The movie shows that Pan-STARRS is only visible shortly after sunset, when it is located low above the Western horizon.

The movie was created using my Python visibility-maps module. The original frames that went into making the movie can be found here and may be used freely.

Answer: more common than you might think!

The population of Solar System bodies which cross Earth’s orbit range from mm-sized dust (producing meteors) to km-sized asteroids (producing mass extinctions). The frequency of such objects is constrained by meteor observations on one hand (there are ~1000 visible meteors per second across the planet), and asteroid surveys on the other hand (a 10km body will hit us every ~100 million years).

The object that struck Russia falls somewhere in the middle of this range. We know that a meteoroid needs to be larger than 1 meter in diameter to penetrate deep enough into the atmosphere to cause a significant airburst (though velocity, entry angle and composition are important too). At the same time, the scale of destruction is a lot smaller than the famous Tunguska event in 1908, which is thought to have been caused by a 50m-sized body. Hence a first guess for the size of the Russian meteoroid would be “between 1 and 20 meter”.

We don’t know the frequency of asteroids in this size range very well. There are not enough meteor observing cameras to detect these rare events, yet the objects are too small and faint to be detected by telescopes ahead of their impact (apart from one notable exception in 2008).

There is an industry that is very successful in detecting these impacts however; military space surveillance. The US Defense and Energy departments operate satellites to detect the heat signatures from nuclear weapons and rocket launches from space. In 2002, a team led by Professor Peter Brown obtained access to classified data on 300 large fireballs detected between 1994 and 2002 by the military. The authors combined this information with ground-based observations to estimate the relationship between the size of meteoroids and their impact frequency:

Credit: Brown et al. 2002, Figure 4.

The work by Brown et al. may roughly be summarized as follows:

• a 10cm-body hits us every few minutes;
• a 1m-body every few months;
• a 10m-body every few years to decades;
• a 100m-body every few millenia.

Hence, a fireball like the one in Russia is likely to occur somewhere between every few years and every few decades, but the uncertainty is large. Shifting the trend slightly upwards or downwards can change the estimated frequencies by a factor of several, and so we should be careful to pin down any numbers with large confidence.

Whilst the Russian meteor appears to be the largest recorded event for a century, it would be imprudent to infer that it hence only occurs once a century. It is difficult to rely on historical records of the past, because less than 20% of the surface of our planet is inhabited, and so many impacts might have gone unnoticed. In fact there is some evidence for a possible ‘Brazilian Tunguska’ on 13 August 1930 and a ‘Guyana Tunguska’ on 11 December 1935, but events like these may have been undocumented or forgotten.

Moreover, there are reasons to believe that the impact frequencies are not constant, but may be elevated during certain periods (see my recent talk on this topic). Unfortunately, the US military announced in 2009 that they will no longer share fireball observations with scientists, so we’ll have to come up with other ways to pin down the exact danger coming from small asteroids!

While there are plenty of maps online showing where in the sky you may find 2012 DA14, I could not find any maps showing where on Earth you have to be to get a good view. So I made a few maps myself. Green areas in the animated gif below indicate parts of the world where the asteroid will be above (and the Sun below) the horizon as it sweeps past. The maps were generated using a Python class which I pushed to my GitHub repository.

For years I have employed the popular strategy of sticking random files on the desktop, and whenever it becomes a mess, create a folder called “oldstuff” and move everything into it. The strength of this strategy is that it can be repeated indefinitely (oldstuff2, oldstuff3, oldstuff4…), the weakness is that you remain a file-hoarding, domusindexophobic maniac.

In the past few months I decided to adopted a more sensible strategy. From henceforth, my home directory shall never contain more than 7 (non-hidden) directories:

• ~/tmp: all downloads and unimportant files go here; may be emptied without notice.
• ~/bin: locally installed software, i.e. “./configure –prefix=~/bin/foo” and my Python distro.
• ~/dev: source code trees for programming projects (sub-directories are GitHub repositories!)
• ~/proj: various non-programming projects.
• ~/doc: subdirectories for receipts, boarding passes, papers, books, etc.
• ~/lts: things to archive forever (“Long Term Storage”); e.g. photos and science data (with README files!)
• ~/msc: a resort for things which don’t fit elsewhere.

There is just one exception to this rule: the additional directories which are forced upon me by the Apple Homedir Police in OSX are allowed to be present (I hardly have a choice), but they are not allowed to contain anything useful. There is no exception to the rule in any of the Linux boxes I use.

I employ different backup strategies for each folder. For example, parts of ~/lts sit on an external network drive, while much of ~/doc sits on DropBox and most of ~/dev sits on GitHub.

Is this a Sensible Approach™? What does your nest look like?

Footprint of the IPHAS survey.

Let me explain this footprint briefly. The survey uses the Wide-Field Camera (WFC) at the 2.5-meter Isaac Newton Telescope (INT) in La Palma. This camera consists of four thinned 2048 × 4096 pixel CCDs arranged in an L shape like this:

Footprint of a single IPHAS (WFC) exposure.

The four CCDs provide a combined field of view of 0.3 deg², hence the minimum number of pointings required to cover the 1800 deg² survey area is 6000. However, the survey was designed to provide a 5% overlap between adjacent fields to enable data quality and calibration checks. As a result, the footprint contains a total of 7635 fields which partially overlap.

Furthermore, each field was paired with a so-called offset field at 5 arcmin West and South, to cover the narrow gaps between the CCD chips. This explains the final number of 7635 x 2 = 15 270 pointings. When their footprints are plotted using 30% opacity, a complicated sky coverage pattern emerges. Whilst the majority of the sky is covered twice, there are overlap areas which are covered three or four times (dark patches) and narrow bands between CCD chips which are covered  only once (brighter strips):

IPHAS footprints plotted with 30% opacity. The darkest overlap areas are covered four times, while the the narrow bands between CCD chips are exposed only once.

A tricky question which results from this coverage pattern is: how should the final source catalogue be generated? Some objects are observed four times, sometimes many years apart, whilst others are only observed once. Should multiple exposures of a single object be averaged, or should only the best exposure be adopted in the source catalogue? Such tricky decisions have to be made for all surveys because telescopes don’t make catalogues (see Hogg & Lang 2010.)

An obvious solution is to release both a catalogue of individual detections on one hand, and a catalogue of averaged source properties on the other hand (there has already been a fair bit of thinking and discussion in this direction within the IPHAS team!)

Such catalogue releases should preferably be supplemented by scriptable access to the calibrated pixel data, through image cut-outs, to enable users to re-measure properties or fit models to the actual pixel data. Is there a standard VO protocol for this? There is the Simple Image Access protocol, but is not quite designed for cut-outs!

How do you think surveys with a complicated sky coverage pattern should generate their catalogues and make data available?

While the peak was clearly exceptional in terms of radio observations, it remains unclear how many meteors could be seen with the naked eye or video cameras. The time of the peak mainly favoured sparsely populated areas of our planet, and visual observations have so far only been reported by amateur-astronomers Alexandr Maidik in Ukraine (who recorded 55 Draconids between 16h00-18h00 UT) and Jakub Koukal in the Czech Republic (who recorded 60 Draconids between 17h00-19h10 UT).

Using their data, I posted this graph of the Zenithal Hourly Rate (ZHR) on the website of the International Meteor Organization (IMO):

Visual activity curve of the Draconids 2012

While Alexandr and Jakub are both very experienced and reliable observers, two visual reports are never enough to construct a reliable activity profile. This is because different observing conditions introduce different systematic uncertainties in the ZHR estimates, which can only be corrected by averaging a large number of independent observations.

Moreover, both observers suffered from either poor limiting magnitude or evening twilight. The period covered by Alexandr alone yields rates up to ZHR=600+/-200 meteors per hour in the interval 16h50-17h00 UT. However, as soon as data from Jakub is available from 17h00 UT, rates drop to ZHR=160+/-40. When leaving out the data from Alexandr, Jakub’s first interval (17h00-17h10 UT) yields ZHR=104+/-39.

The latter estimate of ZHR ~100 meteors/hour seems broadly consistent with data from the IMO video network, which caught the end of the peak and reported ZHRs in the range of 30 to 60 meteors per hour in the interval 17h00-17h20. This is mainly based on two stations in Hungary, operated by amateur-astronomers Antal Igaz and Zsolt Perko:

Video activity curve of the Draconids 2012 (credit: IMO video network)

On the basis of this visual and video data, it seems tempting to conclude that the outburst was dominated by small meteoroid particles and hence produced only a moderate outburst in the optical (ZHR < 200 meteors per hour?), but more data is required to make such claim.

If anyone is aware of reports from Russia or China about this event, please shout. This area of science crucially depends on data provided by citizens!

Radio observing systems are only sensitive to the smallest meteoroids, which produce very faint shooting stars. It is not yet clear how many meteors were bright enough to be seen with the naked eye or video cameras. However, independent video observers in the Czech Republic and Germany told me they recorded more Draconids than usual in the early evening hours (~10 to 40), which does indicate a significant enhancement (albeit no exceptional storm). Careful analysis will follow.

Note that the time of the peak did not favour most parts of Europe, which were still experiencing evening twilight. Observers across Asia were best placed to observe the event. Green and yellow areas in the map below indicate parts of the world where the sky was dark, and the radiant above the horizon, at 17h00 UT tonight:

This outburst may have been caused by the narrow trail of dust and debris left behind by the parent comet in 1959. Our planet had been predicted to encounter this specific dust trail in 2012 by meteor modeler Mikhail Maslov, but the level of activity appears to have been somewhat higher than expected. This is not a first for the Draconids: similar surprises occured in 1985, 1998, 1999, 2005, and MAJOR storms were seen in 1933 and 1946.

I am inspired by a number of existing research bloggers who I enjoy reading. They include Andrew Gelman, David Hogg, Sarah Kendrew, Andy Lawrence, Adrian Price-Whelan, Robert Simpson, AstroBetter, Remanzacco, Zooniverse Blogs, Stuart Lowe, as well as a a bunch of micro-bloggers on Twitter, and others who I forget to list.

This blog has just two rules:

1. I shall not apologise for the frequency of posts, or lack thereof.
2. I shall not apologise for breaking the first rule.

If you agree to these conditions, then please do subscribe to the RSS feed.