How do you store a universe worth of data? Three SA universities have the answer
We already have the technology to know how the universe was made, but the challenge for scientists now is to store and process the massive amounts of data needed to do this
A gargantuan "research cloud" has been built by three South African universities as the country takes the lead in unravelling the mysteries of the universe.
This became a necessity when the most advanced technology in astronomy — like the Square Kilometre Array (SKA) in the Northern Cape — meant that unfathomable amounts of data from galaxies far away in space and time would need to be stored and processed by scientists or researchers.
"We now have the technology to know how the universe was made‚" says Professor Russ Taylor‚ "but there is a catch: the answer to these questions live in a massive amount of data‚ and in the next decade‚ processing that data is the biggest challenge."
Taylor is the director of the IDIA (Inter-University Institute for Data Intensive Astronomy) which launched the IDIA Research Cloud on Monday at the Iziko Planetarium in Cape Town — a cloud built especially to deal with all the data.
The three universities that make up IDIA are the University of the Western Cape‚ University of Cape Town‚ and University of Pretoria.
"The SKA has created a data monster‚ and now we have to solve that problem. We need researchers to really be able to play with the data‚ which contains so much information‚ but right now we do not know how to tease it out. This democratises the process‚" said Taylor.
Already‚ SKA precursor MeerKAT — which consists of 64 dishes — was launched 10 days ago and is producing data that no average computer could handle.
Dr Fernando Camilo‚ chief scientist at the South African Radio Astronomy Observatory (SARAO)‚ said: "The beautiful images we see of our universe only exist after massive amounts of data have been processed after being collected over hundreds of hours."
For Sibusiso Mdhluli‚ a master of science student at the University of the Western Cape‚ the ever-changing world of astronomy is creating new career paths for young people.
As he sharpens his skills in the world of machine learning‚ he says: "Astronomers used to look at pictures and see what was interesting to them. Now‚ with the technology we have available‚ they take millions of images and a machine can then produce images for them in seconds by feeding all the data into an algorithm."
The transformative technology would soon impact on other industries like medicine‚ he said.