Pretty Mapping of Australia using R

January 5th, 2017

I’ve recently been experimenting with Data Visualisation in R. As part of that I’ve put together a little bit of (probably error ridden and redundant) code to help mapping of Australia.

First, my code is built on a foundation from Luke’s guide to building maps of Australia in R, and this guide to making pretty maps in R.

The problem is that a lot of datasets, particularly administrative ones, come with postcode as the only geographic information. And postcodes aren’t a very useful geographic structure – there’s no defined aggregation structure, they’re inconsistent in size, and heavily dependent on history.

For instance, a postcode level map of Australia looks like this:

Way too messy to be useful.

The ABS has a nice set of statistical geography that will let me fix this problem by changing the aggregation level, but first I need to convert the data into another file.

Again, fortunately the ABS publishes concordances between postcodes and the Statistical Geography, so all I need to do is take those concordances and use them to mangle my data lightly. First, I used those concordances to make some CSV input files:

Concordance from Postcode to Statistical Area 2 level (2011)

Concordance from SA2 (2011) to SA2(2016)

Statistical Geography hierarchy to convert to SA3 and SA4

Then a little R coding. First convert from Postcode to SA2 (2011). SA level 2 is around the same level of detail of postcodes, and so the conversions won’t lose a lot of accuracy.And then convert to 2016 and add the rest of the geography:

## Convert Postcode level data to ABS Statistical Geography heirarchy

## Quick hack job, January 2017

## Robert Ewing

require(dplyr)

## Read in original data file, clean as needed.

## This data file is expected to have a variable 'post' for the postcode,

## and a data series called 'smsf' for the numbers.

data_PCODE <- read.csv("SMSF2.csv", stringsAsFactors = FALSE)

## Change this line depending on your data series.

## This code is designed to read in only one series. If you need more than one,

## you'll need to change the Aggregate functions.

## Change this line to reflect the name of the data series in your file

data_PCODE$x <- as.numeric(data_PCODE$smsf)

data_PCODE$smsf[is.na(data_PCODE$x)] <- 0

data_PCODE$POA_CODE16 <- sprintf("%04d", data_PCODE$post)

## Read in concordance from Postcode to SA2 (2011)

concordance <- read.csv("PCODE_SA2.csv", stringsAsFactors = FALSE)

concordance$POA_CODE16 <- sprintf("%04d", concordance$POSTCODE)

## Join the files

working_data <- concordance %>% left_join(data_PCODE)

working_data$x[is.na(working_data$x)] <- 0

## Adjust for partial coverage ratios

working_data$x_adj = working_data$x * working_data$Ratio

## And produce the SA2_2011 version of the dataset. Data is in x.

data_SA2_2011 <- aggregate(working_data$x_adj,list(SA2_MAINCODE_2011 = working_data$SA2_MAINCODE_2011),sum)

## Now read in the concordance from SA2_2011 to SA2_2016

concordance <- read.csv("SA2_2011_2016.csv", stringsAsFactors = FALSE)

## Join it.

working_data <- concordance %>% left_join(data_SA2_2011)

working_data$x[is.na(working_data$x)] <- 0

## Adjust for partial coverage ratios

working_data$x_adj = working_data$x * working_data$Ratio

## And produce aggregate in SA2_2016

data_SA2_2016 <- aggregate(working_data$x_adj,list(SA2_MAINCODE_2016 = working_data$SA2_MAINCODE_2016),sum)

## and finally join the SA2 with the rest of the hierarchy to allow on the fly adjustment.

statgeo <- read.csv("SA2_3_4.csv", stringsAsFactors = FALSE)

data_SA2_2016 <- data_SA2_2016 %>% left_join(statgeo)

The end result gives you a data set that can be converted to a higher level. Here's the chart above, but this time using SA3 rather than postcodes:


Countdown to Star Trek: Discovery – 164 days to go

December 18th, 2016

Another sliver of news, this time casting for a lead:

Star Trek: Discovery has finally found its Number One: Sonequa Martin-Green. […] She’s Lieutenant Commander Rainsford

Another actor I’m not very familiar with (geek confession: I don’t watch The Walking Dead, it’s just never appealed very much to me). I think the idea of not focusing on the Captain is a good one, but not really as ground breaking as it’s being presented sometimes. After all, TNG really had very few stories that focused on Picard per se, and Voyager had few stories that really focused on Janeway.

There was also another bit of news from earlier in the week which I missed:

CBS All Access’ show has added three new castmembers—all reportedly playing Klingons—to Star Trek: Discovery. This tells us that instead of the traditional rotating cast of Klingons, we’re going to have an ongoing focus on specific members of the Klingon Empire.

This is more in the sigh category for me. Klingons are probably the dullest of the recurring ‘villain’ races, and while I’d love to see a more nuanced approach I suspect that isn’t what we’re going to get. This pretty much confirms my earlier guess about it being in the ‘Klingon Cold War’ period. The real question – will they go for Klingon makeup (a la TNG) or for the ‘augment’ human appearance of TOS (as seen in the very end of Enterprise).


Countdown to Star Trek: Discovery – 179 days to go

December 3rd, 2016

More news, this time a bit more worrying.

Even though it had been said that Bryan Fuller would remain an executive producer for Star Trek: Discovery, Fuller is now clarifying that the only work he will do on season one of the show is what he’s already done.

This suggests that there’s a lot more going wrong behind the scenes than we’ve seen before. While Bryan Fuller was overcommitted with American Gods, the lack of a new clear controlling vision suggests that we might be heading towards the corporate blandness (and failure to understand Trek) of a lot of the worst of Voyager and Enterprise.

No need to panic quite yet, but I am a bit less optimistic than I had been.


Countdown to Star Trek Discovery – 181 days to go

December 1st, 2016

Big news today is some information on casting. We have three roles:

  • Michelle Yeoh as Captain Georgiou. Weirdly, captain of the Shenzhou, not the Discovery. Most famous for Crouching Tiger Hidden Dragon, this is a big name for the cast, although probably a more minor role.

  • Doug Jones as Lieutenant Saru. Has hardly ever appeared on TV without being in some kind of monster makeup, and will be playing an alien again here.

  • Anthony Rapp as Lieutenant Samets. More famous as a broadway star (he was in the initial production of Rent), this will be quite a new thing for him. His character is described as a fungus expert.

Overall – interesting, but hard to see where the show is going from these.


Countdown to Star Trek: Discovery: 257 days to go

September 16th, 2016

What’s that you say? Wasn’t the counter at 171 days a month ago.

News today is that Discovery has been delayed until May 2017. I’m using 31 May as the date until I get better information.

Overall, not too worried about this. With no news on casting or filming a January release had been looking unusual for a bit. And more time normally means higher quality. I’m happy to wait. Especially now they missed the 50th anniversary.


Quick Reviews of the other 17 rides I went on in Orlando

September 10th, 2016

While in Orlando I also visited Kennedy Space Centre, SeaWorld and Universal Studios Florida.

Kennedy Space Centre

Shuttle Launch Experience – using a ride vehicle very reminiscent of Star Tours at Disney, a fairly realistic (or so we’re told) simulation of what it’s like to be in a shuttle as it’s launching. A

SeaWorld

Manta – coaster with a manta ray theme. Good, fast coaster with four inversions and a top speed of 90 km/h. Fun, but a very constricting and odd ride position took away from it for me. A-

Kraken – nicely balanced coaster, with a good use of scenery in a couple of loops. Seven inversions and a top speed of 105 km/h makes for a nice experience, but it also has a good duration (2 minutes). A

Antarctica: Empire of the Penguin – mainly a story ride similar in concept to Haunted Mansion, but using a much more advanced ride vehicle. But the story told is pretty dull, and the technology isn’t really used to good effect. The finale is good, arriving in the actual penguin habitat for SeaWorld. But even penguins can’t rescue this ride. C-

Journey to Atlantis – another flume ride, although a bigger drop (and bigger splash) than Splash Mountain. There’s an attempt to theme things around Atlantis a little bit, but it’s incoherent and hard to follow. A-

Universal Studios – Universal Studios Florida

Despicable Me: Minion Mayhem – as weak a story as the Despicable Me, but an interesting use of simulator motion technology in a theatre. It worked surprisingly well. B+

Hollywood Ripe Ride Rockit – no discernible theming, just a fairly extreme coaster with a vertical initial climb and near vertical drop. Ultimately just a bit ‘loud’ and not that impressive. C

Revenge of the Mummy – dark coaster with some simulator ride elements. Most impressive part was the use of fire effects (I think I could have roasted a marshmallow on them), but ageing film projections took the shine off. B-

Transformers The Ride: 3D – my biggest surprise, was expecting a pretty mediocre ride (consistent with the movies), but instead got a a really clever combination motion simulator (a la Soarin’) and coaster ride. Good use of effects, and good scene transitions made up for a general lack of coherence in the plot. A

The Simpsons Ride – motion simulator in a projection dome. Showing its age a little, but still quite well done, with the motion simulator given a fair workout. Writing is clearly authentic to the show (good), but mainly the later years (bad). B+

MEN IN BLACK Alien Attack – pretty dull ride only slightly enlivened by being given a gun to shoot at the aliens with. Not much thrill, and the age of the franchise is showing. D+

Harry Potter and the Escape from Gringotts – good ride technology, and a reasonably fun ride. Links in directly to the Harry Potter films, but without really making a lot of sense. B

Hogwarts Express – I thought this was just the transfer from one park to the other, but it’s actually far more themed than I expected. Clever use of projections provide a good illusion on being on the Hogwarts Express. A

Universal Studios – Islands of Adventure

Harry Potter and the Forbidden Journey – the best of the Harry Potter rides at Universal. Starts with an incredibly well themed line through Hogwarts, and then onto one of the most advanced ride vehicles around. Similar to Transformers 3D in its combination of motion simulator, coaster and projection domes, but with a (slightly) more coherent story and more movement from the simulator. A

Dragon Challenge – the weakest of the Harry Potter rides, really just a lightly rethemed existing coaster. It’s a reasonable coaster ride, but doesn’t have any real spark. C

Jurassic Park River Adventure – river boat ride with a Splash Mountain style drop. The dinosaurs are just embarrassing, you can see where the rubber is sagging. And the drop itself doesn’t get you nearly wet enough (another hot day you see). C

The Incredible Hulk Coaster – pretty good coaster, with a nice high speed launch. But I found myself banging my head from side to side onto the restraints as it spun around, leaving me walking off the coaster feeling a bit battered. C+