Australian Census 2016 - rank incompetence, lies and a failure of outsourcing

The conduct of the Australian Census 2016 is a worrying demonstration of government incompetence.  Here are my observations on the debacle.

Privacy concerns

In the lead up to the Australia's 2016 Census serious concerns were raised relating to a change in the retention of personal data that allows respondents to be identified and what this would be used for.

I didn't see an adequate response from the Australian Government or the Australian Bureau of Statistics about why this was being done and what the personal identification data links would be used for.

I have concerns that government departments and other authorities could access this data and use it to track down issues with individuals, which is not the purpose of census data.


Funds stripped from ABS
The Rudd and Abbott governments both cut funding for the ABS.  This set the scene for the debacle that was to follow.  The ministers previously involved in this were Kelly O'Dwyer and Joseph Ciobo.


Online data collection outsourced to IBM
In a apparent attempt to save money a decision was made - its not clear to me by which minister - to shift the majority of data collection to an online process using the Internet, and to outsource the technology solution to IBM.

With this outsourcing, it was vital that requirements for the solution be specified by government, including:

• Appropriate security for online submission of forms
• Appropriate security for data retained
• Performance - the maximum number of concurrent users supported
• Website availability - protection for Distributed Denial Of Service (DDOS) attacks

Statements were made by the ABS and the newly appointed minster Micheal McCormack that everything was in order, data would be secure and the website would handle the number of user that would use it on the night Tuesday 9 August 2016 (data collection night).


Website meltdown and failure
During the evening of Tuesday 9 August 2016 people experienced problems accessing the website.  Some people completed forms but were not able to submit them.  Others couldn't access the website at all.  Messages via Twitter from the ABS were confusing - they said to "try again later".  Millions of people gave up trying to submit their response and were left wondering what had happened.


The morning after - claims of hacking
There was intense interest on Wednesday morning on what had happened. Claims emerged on ABC morning radio that

"the website had been hacked"

"no data has been compromised"

"there was a DDOS attack" (no evidence has been provided for this)

"DDOS is not actually an attack because no data was accessed" (by Minister McCormack)

"the website was taken down by the ABS due to a false positive alert from IBM"

"a hardware router failed that prevent people accessing the website"

"the website was tested for up to 1 million concurrent users"

It is not possible to determine the veracity of any of the above claims as no information is available to validate them.

More recently, it has been claimed that:

"access to the website is geo blocked" (you must be in Australia to access the website)

"DNS servers outside Australia were blocked (preventing them routing access requests to the website)

What these claims highlight is rank incompetence by the ABS, the Government and IBM.


Solutions
I work in IT.  The following solutions were possible, but apparently neglected.

Robust security:  Encrypted sessions to secure data (appears to have been implemented)

DDOS protection:  Mechanisms are available to identify and avoid DDOS attacks, which are quite common.

8 million concurrent users:  Website performance should have been scaled to meet up to 8 million concurrent users as the majority of the population is on Eastern Standard Time and therefore was trying to access the online form at the same timee.

Drop the extended data retention:  In the absence of valid reasons for retaining identification data longer, this should be dropped.

Hardware failure: Redundancy and fail over is required, preferably via virtual devices rather than physical ones.

Increase scalability: If the application has been written properly (it may not have been) and the solution is cloud hosted, then performance can be scaled up (e.g. by instantiating more virtual servers) as required, and scaled down when not required.

Unfortunately, recent comments from Prime Minister Turnbull, Minister McCormack and the ABS only amount to misinformation, blame shifting and finger pointing.

The Australian online census meltdown is a failure of outsourcing and reveals gross incompetence of the Australian government.

Turnbull has said that "heads will roll". Perhaps Turnbull and McCormack are the ones who should be sacked for rank incompetence?

This debacle makes a complete mockery of Turnbull's "innovation agenda".

Misinformation, blame shifting and finger pointing won't fix the problems.

See also

• IBM And The ABS Census: Let The Blame Games Begin | Lifehacker Australia
• Lesson of #CensusFail: continued funding cuts mean agencies can’t do their job | The Guardian 
• Census debacle puts the Prime Minister on notice | Sydney Morning Herald 

The Speaker's resignation reflects a government that is in chaos

This tweet by Tony Abbot on 24 Nov 2011 applies now to his government in August 2015.  The Internet never forgets.

The Speaker's resignation reflects a government that is in chaos. The Govt has lost its way, lost its majority and now lost its speaker.

— Tony Abbott (@TonyAbbottMHR) November 24, 2011

Submission: Amendments to the Racial Discrimination Act 1975

Human Rights Policy Branch
Attorney-General's Department
3–5 National Circuit
BARTON ACT 2600
via email

Dear Sir/Madam,

I strongly opposed the proposed amendments to section 18C the Racial Discrimination Act 1975.

The safeguards provided by the Racial Discrimination Act have been in place for almost 20 years, including during the 11 years of the Howard government, giving targets of hate speech a peaceful and legal avenue of redress.

These laws have helped resolve hundreds of cases that would otherwise have been left to fester and to degrade social cohesion.

These laws protect all Australians against racial vilification, not only minority groups, and are one of the few inhibitors we possess against the racism which underpins many overseas conflicts.

I believe that the proposed changes, if passed, will send a dangerous signal that hate speech is sanctioned as a form of freedom of speech, that bigotry has a place in our society.

I do not believe that it is OK in our society to behave or speak like a bigot.

The proposed amendments will encourage those so inclined to take bigotry into the public domain. Even in situations of unambiguous abuse, the victim will be required to prove that the abuse may incite a third party to racial hatred – an extremely difficult test to satisfy.

Those who bring diversity to our country will be more susceptible to racist taunts aimed at their culture, their tradition, their faith, their skin colour. They will be rendered vulnerable to hate speech.  Their protection against this will be compromised.

Our government has a duty to make racism socially unacceptable and to provide the targets of racism with a legal course of action. The proposed changes will take our society in the opposite direction.

I along with over 200 ethnic communities across Australia, including indigenous Australians, oppose the proposed amendments.

I urge the Federal Government to withdraw its Exposure Draft of the Freedom of Speech (Repeal of Section 18C) Bill 2014.

Peter Campbell

Information on how to make a submission via mail or email is available here

An opportunity for a new form of government

The 2010 Australian Federal election results are not yet finalised, but it appears that no party has enough seats (76) to form government on its own.  This situation has not occurred in Australia since 1940.

The Greens have won their first ever lower house seat at a general election (Adam Bandt in Melbourne).

The three previous (incumbent) independents have been returned to office.  These are:

• Tony Windsor, New England (rural NSW)
• Bob Katter, Kennedy (rural QLD)
• Rob Oakeshott, Lyne (rural NSW)

In addition, it is quite likely that Andrew Wilkie may win the seat of Denison in Tasmania as an independent.

Negotiations are in progress between the three confirmed independents and both the Labor party and the Coalition as to how a minority government might be formed.

I think this is a good outcome for democracy.  All those elected (all parties and independents) have been selected via the current electoral process by the people of Australia.  It is incumbent on them to form a stable and effective government.

These three confirmed independents have stated that a new form of government will be required to provide the stability required, and that traditional party politics should be shelved to make this happen.  I agree.

If either major party forms government in their own right they tend to run their own agenda along their party line rather than respecting the best interests and wishes of the Australian people.  They are basically accountable to nobody until the next election.

We saw this with the Rudd Labor government ignoring the recommendations of the extensive Garnaut Review of Climate Change and concocting a fatally compromised Emissions Trading Scheme (the CPRS), that was initially supported by the Coalition opposition, then opposed.  It failed because it was no good.

The Henry Tax Review finished early in 2010 was eventually released by the Rudd Labor government, who then chose to implement only 2 of the 137 recommendations (the mining tax being one of them) in the midst of an election campaign for political reasons.

Rob Oakeshott made the point on the 7:30 Report (special election edition 22/8) that a lot of time, money and effort has gone into these and other similar reports, which could be considered by the next government with more care and attention than the previous one.   In short, the next government should use this type of information to formulate policies for the future covering energy, carbon pollution, taxation and water utilisation and conservation, rather than just playing short term political games about these important issues.

It seems that the old political parties have become part of the problem contributing to lack of action on climate change and inadequate planning and investment in infrastructure for the 21st century.  They are stuck in old paradigms of winning, losing, being "in government" or "in opposition".

Why should 51% of our elected representatives be given the right to "govern" in an autocratic manner with the other 49% consigned to "opposition" where they spend most of their efforts whining, criticising, attacking and just opposing for the sake of it?

If Malcolm Turnbull would be a better treasurer than Wayne Swan, why shouldn't he get the job?   Our current political system totally precludes this (for this example with a Labor Government in office).

The Labor, Liberal and National parties are out of touch and out of date.  The Greens need to be very careful they don't end up in the same state.

Tony Abbott seems to think he has won the election and Labor has lost, apparently oblivious to the reality that the Australian people have given him no mandate to govern.

Julia Gillard seems to be adopting a better negotiation approach to possibly forming a minority government with the support of the independents and the single Greens lower house member.

I think we need a form of government where all 150 lower house MPs are accountable for delivering stability, innovation, good management of the executive arm of government and planning for a prosperous and sustainable future.  Bring it on please.

External links

• Parties tussle to form government, The Age
• Greens hedge their bets, The Age
• Labor leads race for minority government, The Age
• Hung parliament looms, as voters punish Gillard, The Age
• We deserve to lead, says Abbott, Heraldsun
• Politics abhors a vacuum, ABC

Senator Kim Carr happily subsidises fossil fuel but not electric vehicles

It is very disappointing that the Australian Government continues to provide massive subsidies for fossil fuels - around $9 billion per year - while Senator Kim Carr has ruled out any subsidies for electric vehicles.

From the Sydney Morning Herald:

The federal government instead believes the future of the car industry lies in the development of existing technology across petrol, diesel and LPG engines.

''It's not our intention to run programs to support any particular form of technology,'' Minister for Innovation, Industry, Science and Research Kim Carr said.

This is yet another example of the failure of the Australian Government and our elected representatives to provide leadership towards a low carbon future.

Electric vehicles can reduce our carbon emissions if they replace fossil fuel powered vehicles, and they can achieve zero emissions when the source of electricity for recharging them is 100% renewable energy.

This is also a missed opportunity to develop an electric vehicle that could be exported around the world.  China, France, Japan, the United States, Germany, Korea and India are all building electric vehicles.

Kim Carr's so called "green car fund" spends $1billion of Australian taxpayers money on very dubious projects such as a "green 6 cyclinder motor" (Ford Australia).

Not surprisingly, the oil companies in Australia regard their profits from fossil fuels to be "safe for 10 years".

Kim, the government should subside the development of low emissions technologies.  You should also remove all subsidies on fossil fuel use.

Transport plans for Melbourne and sustainability

I am away for a bike race this weekend - the Tour of Bright.

I have just read about what is likely to be in the Brumby Government's transport plan due for release.

What a disappointment. It seems the government will avoid once again any signficant investment in public transport and keep building more unsustainable roads and freeways.

The bad

• Eddington freeway - Port to CityLink (freight, road). It may be needed, but more trucks and cars rather than rail.
• Western Ring Road freeway connection investigation - to ruin the green belt in Greensborough, Diamond Creek and Warrandyte. More Los Angeles style "freeways to everywhere".
• Freeway bypass around Frankston - having just constructed the unneeded and poorly used Eastlink, we now need to "extend the freeway" so that the precitable bottleneck at Frankston is eased, to ensure profits for Eastlink.

The ordinary

• A very expensive $8b rail tunnel connecting Caulfield to Footscray, but this is deferred. The cynical might say it will never be built. I am not convinced it is needed, or should be a priority. Underground rail is 20 times more expensive than on the ground rail.

The good

• South Morang rail extension. Long overdue; if it happens it will be the first significant rail line to be built since 1930.
• Tarneit Rail link to growing western suburbs.
• NO Clifton Hill - Footscray road tunnel. It is not needed, would have been hugely expensive, and of course just encouraged more car use.

The missing

• No proper inner city metro for Melbourne
• No rail for Rowville and Monash University (promised for years)
• No rail for Doncaster (promised for years)
• No airport rail link (still thousands of cars and taxis every day, and hundreds of buses)
• No rail (light or heavy) for Melbourne's recent boundary expansions just announced by the Victorian Government (West and North of Melbourne)

I give the Brumby Government about 2/10 for this myopic and half baked transport plan. It was doomed to not deliver given the very restricted "East West Needs Assesment" brief that Eddington was given.
And of course, there will be no consideration or measurement of carbon emissions for transport options - which will be much higher for road and freeways.
In summary, the Victorian Goverment's transport planning is a bad as their water management strategy. Ill considered, focused on industry lobby groups, and basically ignoring sustainable transport options that will improve Melbourne's liveability. Thumbs down.

Al Gore on Climate Change Leadership

It is imperative that the Australian Government work constructively with other nations via the United Nations on a post-Kyoto agreeement to tackle climate change that includes tangible emission reduction targets. The current approach endorsed by John Howard, Alexander Downer and Malcolm Turnbull to set "aspirational voluntary targets" will be ineffective.

Self regulation of the most polluting industries carries a very real risk that short term profit motives will outweigh taking real action on addressing climate change - which science and our own experience now tells us is one of the greatest challenges facing the world today.

Here is a good quote from Al Gore on the important role governments have in setting real and binding emission reduction targets globally.

Quote of the Day: Al Gore on Climate Change Leadership, Montreal Protocol
(From TreeHugger)

All of the market initiatives are incredibly important. The market allocates more money in one hour than all of the governments allocate over a year's time. But governments set the rules of the road and determine how markets allocate capital and make decisions. And there should be no mistake that this crisis, the climate crisis, is not going to be solved only by personal action and business action. We need changes in laws; we need changes in policies; we need new leadership and we need a new treaty. We need a mandate at Bali during the first 14 days of December this year to complete a treaty not by 2012 but by 2009, and put it completely into force by 2010. We can do it and we must do it. ...

We face a genuine planetary emergency, we cannot just talk about it, we have to act on it, we have to solve it, urgently. ... Last week the world celebrated the 20th anniversary of a great success story. A hole in the ozone layer was discovered in 1985. And then, in the following year and a half, action took place. Some people said voluntary action will solve it; businesses will take the initiative. The Secretary of the Interior at that time said voluntary measures like wearing more sunglasses and floppy hats was the answer.

I would like to call on President Bush to follow President Reagan's example and listen to those among his advisers who know that we have to have binding reductions in CO2; we have to put a price on carbon, and the United States of America has to lead the world to solve the climate crisis."

—Al Gore, former vice president of the Untied States, in the opening plenary session of the Clinton Global Initiative, 26 Sept. 2007