Archive for the 'Economics' Category

Pretty Mapping of Australia using R

January 5th, 2017

I’ve recently been experimenting with Data Visualisation in R. As part of that I’ve put together a little bit of (probably error ridden and redundant) code to help mapping of Australia.

First, my code is built on a foundation from Luke’s guide to building maps of Australia in R, and this guide to making pretty maps in R.

The problem is that a lot of datasets, particularly administrative ones, come with postcode as the only geographic information. And postcodes aren’t a very useful geographic structure – there’s no defined aggregation structure, they’re inconsistent in size, and heavily dependent on history.

For instance, a postcode level map of Australia looks like this:

Way too messy to be useful.

The ABS has a nice set of statistical geography that will let me fix this problem by changing the aggregation level, but first I need to convert the data into another file.

Again, fortunately the ABS publishes concordances between postcodes and the Statistical Geography, so all I need to do is take those concordances and use them to mangle my data lightly. First, I used those concordances to make some CSV input files:

Concordance from

Postcode to Statistical Area 2 level (2011)

Concordance from SA2 (2011) to SA2(2016)

Statistical Geography hierarchy to convert to SA3 and SA4

Then a little R coding. First convert from Postcode to SA2 (2011). SA level 2 is around the same level of detail of postcodes, and so the conversions won’t lose a lot of accuracy.And then convert to 2016 and add the rest of the geography:

## Convert Postcode level data to ABS Statistical Geography heirarchy

## Quick hack job, January 2017

## Robert Ewing


## Read in original data file, clean as needed.

## This data file is expected to have a variable 'post' for the postcode,

## and a data series called 'smsf' for the numbers.

data_PCODE <- read.csv("SMSF2.csv", stringsAsFactors = FALSE)

## Change this line depending on your data series.

## This code is designed to read in only one series. If you need more than one,

## you'll need to change the Aggregate functions.

## Change this line to reflect the name of the data series in your file

data_PCODE$x <- as.numeric(data_PCODE$smsf)

data_PCODE$smsf[$x)] <- 0

data_PCODE$POA_CODE16 <- sprintf("%04d", data_PCODE$post)

## Read in concordance from Postcode to SA2 (2011)

concordance <- read.csv("PCODE_SA2.csv", stringsAsFactors = FALSE)

concordance$POA_CODE16 <- sprintf("%04d", concordance$POSTCODE)

## Join the files

working_data <- concordance %>% left_join(data_PCODE)

working_data$x[$x)] <- 0

## Adjust for partial coverage ratios

working_data$x_adj = working_data$x * working_data$Ratio

## And produce the SA2_2011 version of the dataset. Data is in x.

data_SA2_2011 <- aggregate(working_data$x_adj,list(SA2_MAINCODE_2011 = working_data$SA2_MAINCODE_2011),sum)

## Now read in the concordance from SA2_2011 to SA2_2016

concordance <- read.csv("SA2_2011_2016.csv", stringsAsFactors = FALSE)

## Join it.

working_data <- concordance %>% left_join(data_SA2_2011)

working_data$x[$x)] <- 0

## Adjust for partial coverage ratios

working_data$x_adj = working_data$x * working_data$Ratio

## And produce aggregate in SA2_2016

data_SA2_2016 <- aggregate(working_data$x_adj,list(SA2_MAINCODE_2016 = working_data$SA2_MAINCODE_2016),sum)

## and finally join the SA2 with the rest of the hierarchy to allow on the fly adjustment.

statgeo <- read.csv("SA2_3_4.csv", stringsAsFactors = FALSE)

data_SA2_2016 <- data_SA2_2016 %>% left_join(statgeo)

The end result gives you a data set that can be converted to a higher level. Here's the chart above, but this time using SA3 rather than postcodes:

The exchange rate according to today’s Apple announcements

October 21st, 2009

Apple announced some shiny new things today. Given that the Australian dollar is around 92 cents, we’d hope for a good exchange rate. What has Apple actually done?

  • iMac 22-inch: US $1,199, AUS $1,599

  • iMac 27-inch: US $1,699, AUS $2,199

  • Mac Mini Base: US $549, AUS $849

  • MacBook: US $999, AUS $1,299

  • MagicMouse: US $69, AUS $99

  • Apple Remote: US $19, AUS $25

It’s important to remember that the US prices are before sales tax, so I’ve taken the GST (10%) off the Australian prices to work out the exchange rates.

When we do that, the exchange rate ranges from 71 cents (on the Mac Mini, which is sad because that’s the one I want to buy) to 85 cents on the MacBook. The average is pretty much 81 cents. Clearly there’s some rounding going on to hit nice price points.

Why is it so much lower? Because Apple (like most other companies) isn’t silly. They know that the exchange rate fluctuates, so they don’t set prices based on what it is this week. Rather they look at longer run averages. And if you look at the average exchange rate over the last six months (excluding October, as they would have set prices a few weeks ago) it’s also around 81 cents.

Hands up if you can see the problem

February 27th, 2008

Net Neutrality is one of the biggest hot button issues among the nerd illuminati of the Internet right now. The simple question is whether all internet bits are equal, or should ISPs be allowed to privilege some bits (from their customers or people who pay them) over others.

There are some side issues here, but a big part of it is peer to peer. Which brings me to this story from today that online video distributors can save a lot of money by using peer to peer protocols

In the example given, Democracy Now saves $1,000 (of a $1,200 bill) by using BitTorrent. My question is – who ends up paying that $1,000? If we assume (and it’s not a great assumption) that everything is competitive, then that $1,200 represents the cost of pushing that many bits to end users. If it goes down, then it must mean that $1,000 worth of bits are now being pushed by someone else – in this case, the upstream bandwidth from the users.

So who pays?

At first, probably the ISP of the end users. Their bandwidth out gets used up, costing them money.

They’ll pretty quickly pass that on to the end users. Which means they’ll increase prices for everyone.

So what’s DemocracyNow really doing here? They’re pushing the costs of distributing from themselves on to end users. Which, due to the way pricing is set up, will be borne equally by everyone, regardless of how interested they are. In fact, people who have no interest at all in the video probably end up paying for this too.

I’m not arguing against net neutrality – there are other reasons why it’s a good idea. This is probably more an example of how the pricing for internet access is set up wrong – flat rate charges create strange incentives across the Internet, not just for the end users.

But that $1,000 saving? That doesn’t exist. You’re just making other people pay it.

Why isn’t the computer game business more like films?

February 26th, 2008

Just last week Electronic Arts offered $US 2 billion to purchase another publisher, Take-Two.

This is part of a continuing trend in the computer video game business, with the really big publishers consolidating. In some sense, this is a lot like the film business – the big studios make up a very substantial proportion of the total turnover of the industry.

But the big difference is with the next level down. In the film industry almost everyone is a contract player – directors, writers, and actors all move from studio to studio, only settling at one studio for the amount of time to make one or two projects. But in the games industry most of the ‘talent’ is permanently employed, staying with the company for many years.

This is odd – in many respects the requirements are the same. Video games are expensive undertakings these days – $10-20 million to produce, millions more to market and distribute. This is still well short of the cost of a major movie, but the gap is closing.

In fact, the structure of the video game industry looks a lot like the movie industry of the 1930s. The studio system lasted until the vertical integration of the industry was stopped and a single tycoon (Hughes) stepped in. At present, much like the 1930s film industry, the video game industry has a few stars (which, in this case, are the intellectual properties like Halo or Quake), and most people in the industry are relatively anonymous.

So what will change? My guess is that it will be the rise of the talent.

At present there isn’t a single famous video game writer, and only a couple of famous ‘directors’ (the analogy isn’t perfect, of course). In fact, if you look at the industry’s Game Developer’s Choice Awards the nominations for writing, art direction and so on only mention the game, not the actual writer!

That’s slowly changing. The enthusiast press has been paying attention to the project heads for a while, and is starting to pay a lot more attention to the writers as well.

Once people know the names, the names can ask for more money. A few people can do this now, but as the media pays more attention, more people will become famous (at least in the video game world), and they’ll start to move from project to project in search of better money.

And once that happens, then the game publishers are going to start to look a lot more like Hollywood – they own the IP for some of the series, and they bankroll the whole thing. But the people making the games aren’t usually tied to any one publisher, and they move around a lot more than is the case now.

And this will be a really good thing for games, because the best talent will be recognised appropriately, and the best projects will attract the best people.


February 16th, 2008

Valleywag, before per-view incentives were provided to staff:

[Valleywag], after (links are not safe for work…):

Any questions?

Rational economics

January 24th, 2008

I was listening this morning to a recent episode of the great podcast Skepticality, and I was very struck by a question host Swoopy asked of interview-ee Michael Shermer, talking about his new book on economics and psychology:

What do we do […] to make better rational choices and fewer emotional ones.

Dr Shermer gave a good answer about being aware of the tricks marketers play and the findings of economic psychology.

I have a slightly different answer: why should we?

There’s a lot of talk around (especially in the Australian media) about how experimental economics is showing “people aren’t rational”, of limits to rationality. Some of this is very good and interesting. Part of the problem is the word ‘rational’. When most economists use it they’re talking about a very narrow technical definition, that has little to do with the other dictionary meanings. That confuses a lot of people.

But there are also a lot of value judgements tied up in most people’s view of rational. For instance, I want to lose weight, but I also want to eat that chocolate bar. Is it ‘irrational’ if I do eat the chocolate bar? Of course not, it just reflects my preferences or discount rates at the time.

So, if our emotions would lead us to choose one thing, but the ‘rational’ choice is something else, is there any reason to think that it’s always better to choose ‘rationally’? Sometimes, sure. If it makes sense to go to another store for $50 off a $100 iPod, it also makes sense to do it for $50 off a $10,000 TV. But the chocolate bar is still a perfectly reasonable choice to make, even if it’s not what you’d choose from some other circumstance. Preferences don’t have to stand still all the time for people to be rational.

Have you looked behind the couch?

January 22nd, 2008

A particularly annoying species of headline writer has been out and about lately. That’s the one who likes to write headlines like:

Stock market loses $10 billion.


It’s silly, lazy journalism. The economy hasn’t ‘lost’ any money. Any losses are either purely theoretical (gains that were never realised), or are offset by the gains of some other party. Sure, some people may have a share portfolio worth a bit less today, but it still owns the same share of the same companies, with the same capital and assets. The market has just decided it’s worth a little bit less now.

No money has gone missing – bank accounts are just the same, other than the people who had to cover their margin calls. But all that money just went straight back into someone else’s bank account.

The reason this is bad journalism is that it distracts us from the real story. People think ‘oh no, a lot of money has been lost’. But really there’s nothing bad about that in itself. The real question is why the stock market went down.

And because the answer for that is mainly ‘fears of a US recession’, that’s bad news for everyone. Telling a story about the massive losses on the stock market creates an illusion that it’s just a problem for ‘rich people’. But the underlying causes of the fall might be a problem for everyone.

Why can’t we all just get along.

January 6th, 2008

In case you hadn’t noticed, the TV and film writers in the US are on strike. It’s been about 9 weeks now since they went on strike in November.

The economics side of this is the question of ‘why’? The standard theory suggests that you should never see strikes, because the simple threat of one should be all that the unions need in order to achieve the best outcome they could. What this ignores is the problem of asymmetric information – not everyone knows the same pieces of information, and some (like how committed the membership of the union is to the strike) are very hard for any side to know perfectly.

But even given that history, strikes in Hollywood tend to be long and vicious. I think that’s probably because the studios can stockpile a lot of finished product, especially in film, and that protects them from the immediate economic consequences. It’s probably also because the personality factors are going to be exaggerated in the environment they all live and work in.

So what are they fighting for?

They’re ultimately fighting over who gets a piece of the pie, and who gets to be treated as a hired player. There’s a strange line drawn in business between the people who are entitled to a share of the profits they create (authors, hedge fund managers), and those who are simple employees who have no rights to a share of profits.

Writers are currently in the first group, getting a (small) share of the profits that come from DVD, video and TV screenings of the things they write. The studios would like to put them into the first group.

Who’s right? It’s hard to say.

The best economic answer to the question of whether you should get a share comes down to bargaining power, and in particular of whether your characteristics (be they creative or simple marquee value) will boost the project more than the next available worker. In Hollywood some writers clearly have that cachet, and some do not. But they all bargain together, and so the value of the top writers gets pushed down to the whole pool to some extent.

So the pure answer would be ‘if they’ll give it to you, then you deserve it’.

Hence the strike.

I just wish they’d get back to it. I’m starting to miss TV, and I’ve finished half my computer game pile…

The scariest thing in economics

November 22nd, 2007

You know what phrase is most likely to scare an economist? “I’m an engineer, and have some thoughts about economics”. Something in the training of engineers seems to make them think that they can apply their knowledge to a completely different field.

A prime example from the National Review talking about the falling dollar.

After constructing a lovely straw man (go marvel at the craftsmanship!) this engineer starts in:

Economic transactions involve the exchange of “something” for “money.” The “something” is specified in terms of number (1, 2, 3, etc.); length/area/volume (“the foot”); weight (“the pound”); and/or time (“the second”). “Money” is specified in terms of “the dollar.”

This is quite simply completely and utterly wrong. There is nothing in all of economics that requires money to be part of a transaction. Economics says that two parties will engage in a transaction when they both receive something of value to them. Back in the days before currency, perhaps I’d trade two pigs to you for a cow. I’m only willing to accept the cow in exchange for the pigs because it has value. So the setup is wrong here from the start: it’s not “something” and “money”, it’s “value” and “value” – the trade is symmetrical.

The problem with this scheme is that the magnitude of our fundamental unit of market value, “the dollar,” is not defined. Being undefined, the value of the dollar can change. This fact gives rise to huge economic costs and risks for which there are no offsetting benefits.

This is true. But I don’t trade money with you because I have an intrinsic value of money. I trade money for goods because I value money. Why value money? Because other people also value money and will trade me goods for it in return.

There’s a kind of collective illusion going on here. It only works because we all sit down and agree that it will work. There’s no guarantee that it will, but somehow our collective delusion holds. We could go back to exchanging everything in barter, but that would cost us a lot of time and effort.

Can the value of the dollar change? Sure, it might. And in the past it has. That’s why we need a system to stabilise inflation.

Rather than being rigorously defined like all other units of measurement, the value of the dollar is left to the market. This is an extraordinarily strange way to determine the magnitude of a basic unit. For one thing, the absolute magnitude of a fundamental unit doesn’t matter — what matters is that it be precisely defined and unchanging. The market also has absolutely no way of determining what the value of a unit “should” be, whether that unit is the foot or the dollar.

Here’s where we see the logical fallacy at full flight. There’s a desire for a nice, absolute resting point – some central value that is determined. But it can’t ever happen. We trade things because we value them. But we’re also human and unique – why should I value anything the same amount as you? And if money is simply a medium ultimately used for buying goods of value, then why on earth should money have any kind of intrinsic value at all?

(Economists who’ve dealt with engineers of this ilk already know where this one’s going at this point).

Having had the job of determining the value of the dollar thrust upon it, the market does the best it can. Moment by moment it equilibrates the supply of dollars and the demand for dollars at some market value. This market value is reflected in the price of gold.

Ah ha! And here we are, another gold bug. Told you the economists could read this play a mile off.

This is really simple to answer: why on earth would gold hold any kind of intrinsic value at all? It’s not that useful a metal, really. It’s not clear how much there is, but there certainly isn’t a fixed supply yet because we haven’t finished digging it all out of the ground. Sure it’s shiny, but plenty of things are shiny. So why gold?

What’s that you say?

Because other people value it too?

Simply put, there is almost no difference between so called ‘fiat’ currency and gold backed currency. There are a few technical points relating mainly to international trade and monetary policy, but in terms of day to day use nothing at all.

And what about the mighty Federal Reserve? Well, the Fed could exercise absolute control over the value of the dollar since it controls the supply of dollars. The Fed has the power to vary the size of the monetary base from zero to infinity. Given this, the Fed has the power to fix the dollar’s value (in terms of gold or anything else) at any level it chooses.

However, the Fed doesn’t do this. Instead, it exerts a vague influence over the size of the monetary base by targeting the federal funds rate.

Now isn’t the time to explain about open market operations, and the way monetary policy works. It’s pretty technical, and not for the non-specialist really. But this is exactly wrong: by changing the interest rate the bank is changing the money supply. It’s just doing it in a fashion that more directly controls the reaction.

A simple analogy might explain: if you wanted to increase the price of some good by 10 per cent, what would you do? Would you reduce the supply of that good until the price went up, or would you just increase the price?

That is, the Fed creates money in response to demand for short-term capital. Given that one use for short-term capital is commodity speculation, and given that commodity speculation is one way to profit from inflation, the Fed is operating a system that is designed to respond to inflation by creating more money.

Such a system exhibits “positive feedback” (like a nuclear reactor) and is dangerous.

We’re now reaching the territory where the lack of knowledge of economics and financial markets is making this very very difficult to understand. While I’m prepared to be corrected by someone with better knowledge of financial markets than me, this appears to be pure gibberish. The federal reserve isn’t doing anything in response to demand for short-term capital – it’s acting in response to unemployment and inflation rates.

Nonetheless, the real problem is the confusion about the fed’s role: the fed does not just reduce inflation, sometimes it wants to increase it. A bit of inflation is a good thing in an economy (just look at what happened in Japan with some deflation).

I think he might be referring to the recent credit crunch, in which case what the Fed was doing wasn’t really increasing the money supply per se, so much as replacing some of the money supply that had disappeared off to non-liquid sources. If they hadn’t acted the money supply would have shrunk, raising interest rates and reducing economic activity. They were just trying to keep things as they were, more or less.

Why would the Fed employ a monetary-control approach that is both indeterminate and dangerous? I believe the underlying problem is an intellectual confusion between “money” and “capital.”

The irony of accusing the Fed of “intellectual confusion” here had me laughing for about five minutes.

Anyway, I’ll skip a few more paragraphs of deep confusion to get to the funniest bits of this article.

A logical definition of the dollar might be “equal in market value to one five-hundredths of an ounce of gold.” The value of all the dollars in the U.S. monetary base would then be maintained by having the Fed’s open-market operations target the price of gold to keep it near $500 per ounce. Because the real market value of gold cannot run away to zero or infinity, the new monetary control system would be determinate and stable.

What I am describing is not a classic “gold standard.” Back then, gold was the monetary base. Instead, the monetary base would be the same “fiat” currency that we have now. Banks would maintain the value of their dollars the way they do now — by redeeming them with the dollars of the monetary base upon demand.

Oh boy.

OK, firstly: “not a classic gold standard”? Rubbish. This is the gold standard with a silly hat. If your dollar is worth 1/500th of an ounce of gold, then it doesn’t really matter if you go to a jeweller rather than a bank to get the 1/500th of an ounce, it’s still a fixed currency. Sure it’s not backed by gold, but the economic effect is exactly the same.

But what’s wrong with this solution? Well, the same problem any fixed currency has, it’s vulnerable to speculative attack. So what would you do? Well, you might continually buy gold to try and force the price up. The Fed would have to sell to you in order to meet its mandate. But the Fed doesn’t have an infinite amount of gold – sooner or later they’ll run out, the price of gold will go up, and confidence in the currency will collapse.

This relates to the earlier suggestion that ‘money supply would be irrelevant’: if there’s more money than gold, then the currency will collapse. So no free banking, no infinite money supply. Too much money and the economy could collapse.

That’s not going to help keep interest rates low, by the way.

This new system would not be concerned with the federal deficit or the U.S. trade deficit. These relate to capital, and capital is not money. Similarly, the system would not be concerned with interest rates, which represent the cost of capital, not money. (As an aside, if the dollar were as stable as the foot, interest rates would be very low.) There would still be a role for the Fed as “lender of last resort,” but this would be a “banking” function separate from monetary control.

Interest rates are the cost of capital, not money???

Just in case you had any lingering thoughts this guy knew what he was talking about, there’s this gem.

Of course interest rates are the cost of money. If I spend money, I’m foregoing the interest on it. There is no definition of cost that can possibly mean that interest rates are not the cost of money.

The distinction between “money” and “capital” here isn’t that silly – but the idea of a ‘cost of capital’ is. There’s a cost for how much money you had to borrow to build the capital. And there’s a return on the capital. In an equilibrium, both of those are equal to the interest rate. The ‘cost of capital’ in economics normally means the cost of building an office block, or making an assembly line.

So, to summarise: this guy is a complete idiot with no understanding of economics whatsoever, or the role of monetary policy. His proposed solution would create massive problems and remove one of the most important arms of economic policy.

That conclusion would be obvious to anyone with first year training in economics. And even if it wasn’t, you might think twice when you realise that everyone with a background disagrees with you (including, by the way, many engineers with actual training in economics).

But for some reason a few engineers seem immune to the kind of self analysis required to actually think that it’s possibly that they’re wrong, and maybe the experts are right. And I don’t know why, but it’s only engineers.

Wait, he’s not done?

Fixed exchange rates between the dollar and (say) the euro are just as desirable as between the foot and the meter. The belief that “floating” exchange rates are needed to deal with trade imbalances between nations represents more intellectual confusion between money and capital.


Wow. Now I need to explain international trade, capital accounts, and the role of the exchange rate?

Nah, too tired for that. This article is already over two thousand words long.

If you want real prosperity, give monetary control to the engineers.

Or, rather, if you’d like to screw the economy, create a massive recession and destroy large parts of the economy, let an engineer look after things. Alternatively, stick with the remarkably effective monetary policy system that has created almost two decades of prosperity and stability in the developed world unmatched in all of history.

To Index or not to Index

July 5th, 2007

As the new financial year rolls around, I’m giving some thought as to how invest my money. I’ve got a mortgage, which means that my investment portfolio (such as it is) is pretty heavily weighted towards property. So I’d ideally like to put some money into the stock market.

As I see it, I’ve got three options:

  1. Put my money into shares directly.

  2. Put my money into a mutual fund.

  3. Put my money into an index fund.

The first option isn’t that attractive for a couple of reasons. Firstly, I don’t have that much to invest, so I won’t be able to get many stocks. So I’d have to be lucky (DJS, I’m looking at you here) to get decent returns at all.

The mutual fund option looks a bit better. There are a lot of options, and I’d be able to invest only a small amount of money at first. But there are two big draw backs:

  • Decent ‘actively managed’ funds charge very high fees, and even then often require large entrance costs. Which means I’d be stuck with the lower tier of funds.

  • Even the good managed funds can’t beat the market: the evidence suggests that pretty much no firms are able to consistently provide higher returns than the market index (e.g. S&P ASX 200). For instance, see Malkiel (1995).

(The other smaller problem is that almost all of them invest in property to some degree, and I don’t want more property investment in my portfolio)

So that pretty much leaves me inexorably with an index fund. An index fund takes shares that give the same return as an index like the ASX200 (usually a pretty substantial subset o the whole index). So by definition the return on the fund is the same as the market. And as an added bonus, index funds tend to be really cheap: about 1% cost of management, compared to over 4% for some mutual funds.

This is pretty much the advice of experienced market analyst Henry Blodget too, who’s written a great book on share investing. I just need to get a copy rather than just reading the free excerpts…