Big Data for Big Questions

Astronomers with big questions like ‘How did we get here’, ‘What was there at the beginning’ and ‘What is out fate’. So it feels inevitable that answering these should be hard, and we should approach the question with caution. After all, we cannot simply believe in our answers, instead we need to agree with evidence-based conclusions drawn from data. As our questions delve us deeper and deeper into these mysteries, we need more and more data. And therein lies the biggest problem facing us today-  how do we deal with such Big Data.

The Guardian newspaper recently published an article on this.

Astronomical data is and has always been big data. Once that was only true metaphorically, now it is true in all senses. We acquire it far more rapidly than the rate at which we can process, analyse and exploit it. This means we are creating a vast global repository that may already hold answers to some of the fundamental questions of the Universe we are seeking.

Does this mean we should cancel our up-coming missions and telescopes – after all why continue to order food when the table is replete? Of course not. What it means is that, while we continue our inevitable yet budget limited advancement into the future, so we must also simultaneously do justice to the data we have already acquired.

Citizen science is one solution. Sites like Galaxyzoo and other projects on simultaneously engage the public and perform a vital scientific role.

But the near future presents a new set of problems..

Thus far, human ingenuity, and current technology have ensured that data storage capabilities have kept pace with the massive output of the electronic stargazers. The real struggle is now figuring out how to search and synthesize that output.20150420-CompletedTMA

The DKI solar telescope in Hawai will produce 15-20Tbyte of data per day, starting 2017. We need to be able to visualiize that, make it science-ready, and then transport it across the internet. As such we are looking at new ways of data mining, machine learning and database systems to help us understand out nearest and star.

It seems that the original science of data, astronomy, has a lot to learn from the new kid on the block, data science. Think about it. What if, as we strive to acquire and process more photons from across the farther reaches of the universe, from ever more exotic sources with even more complex instrumentation, that somewhere in a dusty server on Earth, the answers are already here, if we would just only pick up that dataset and look at it … possibly for the first time.


Touched by the beauty of the Sun. #amacrojot #beauty #sun #aurora

Beauty is a over-used word. But in this case, it’s the only word.

As scientists, we often get asked why we do science.
Is it to help society? Well, we do invent technological advances, medical advances, and engineering advances, but that’s not why do to science.

Is it to make money for the country? Well, we do return investment at the rate of 10:1 over a decade, invent the internet, create new modes of travel, but that’s not why we do science.
Is it to educate? Well, we do teach in critical thinking, mathematics and engineering, but that’s not why we do it.

We do science because nature is beautiful. In fact, the only thing as beautiful as this video is the mathematics we use to explain why this happens. There is no need for the mystical, the magical, or the deities to explain this beauty. This is just mathematical beauty, dancing in front of our eyes.

Great job, Sun.
Keep reminding us why we do this.

A hole in the sun


An extensive coronal hole in the sun’ corona rotated towards Earth over several days last week (May 28-31, 2013). The massive coronal area is one of the largest in a year or more. Coronal holes are the source of strong gusts of the solar wind that carry solar particles out to the Earth and beyond. They appear darker in extreme ultraviolet light images (here, a combination of three wavelengths of UV light) because there is just less matter at the temperatures we are observing in. Solar wind streams take 2-3 days to travel from the Sun to Earth, and the coronal holes in which they originate are more likely to affect Earth after they have rotated more than halfway around the visible hemisphere of the Sun, which is the case here. They have already generated some aurora here on Earth.

Here comes the Sun


The sun has unleashed three strong solar flares since Sunday evening, punctuating a short period of increased solar restlessness that comes as scientists are keeping an eye out for this cycle’s solar maximum. All were X-flares, classified in the category of most intense solar activity. None was pointed at the Earth, though several spacecraft, including the Spitzer space telescope in particular, are in their path.

Each was the most energetic solar flare of 2013 — until the next one came along. The first, an X1.7, occurred around 10 p.m. U.S. Eastern time, on May 12. Then, an X2.8 erupted at 12:05 p.m. on May 13. And the last one, an X3.2, peaked at 9:11 p.m. on May 13. X3.2 is the third-largest flare of this solar cycle, with the largest being an X6.9 in 2011.

“Current estimates are for a 40 percent probability of another X flare during the next 24 hours,” said Alex Young, a heliophysicist and associate director for science in NASA’s heliophysics division. He calculated that each of the flares produced around 100 sextillion (that’s 10 followed by 23 zeros) Joules of energy – the equivalent of 100 million hydrogen bombs – and spat several billion tons of solar material into space. The sunspotted region producing the flares is just rotating into view. Within a week, they’ll be pointed squarely at us – and scientists will be able to get a good look at the turbulent spots, which Young estimates are several Earths across. “We’ll watch it to see if it’s continuing to grow in size, or if it’s starting to get smaller,” he said.

Flares of X-magnitude are multi-stage events, produced by sunspotted regions where magnetic fields are twisted and tangled, knotted and strained like rubber bands wound too tightly. Eventually, strain building up causes the magnetic field lines to snap, releasing a flare. “If you look on the sun, you’ll see some very symmetric-looking, simple sunspots. These are the ones that don’t really do anything, even if they’re really big,” Young said. “But if you look at sunspots like the one that’s been so active, it’s very complicated and very twisted. You would probably see rotation and a lot of jumbled mess. That’s a really strong indication that it’s got a lot of energy built up in it.”

This period of unrest comes as scientists are keeping an eye out for the solar maximum, a period of peak activity that occurs at the height of the 11-year solar cycle. Forecast to be a relatively quiet maximum, the peak should be occurring just about now, though it’s hard to define a peak until it’s passed. An earlier period of peak activity occurred in 2011, but Young suggests that asynchronous activity in the sun’s northern and southern hemispheres can produce twin-peaked maxima. “That’s something we’ve seen in the past, with many different solar cycles,” he said.

Here comes the Sun


A solar belch erupted from the sun early Saturday (Feb. 9), triggering an intense sun eruption aimed squarely at Earth. The solar storm, however, should not endanger satellites or astronauts in space, but could amplify auroras on Earth. The solar eruption —called a coronal mass ejection —occurred at 2:30 a.m. EST (0730 GMT) on Saturday during a minor, but long-duration, flare. It hurled a wave of charged particles at Earth at speeds of about 1.8 million miles per hour (nearly 2.9 million km/h).

The sun has an 11 year cycle, at the peak of which it tends to produce far more solar flares than at at the end of the cycles. This year we are hitting the peak and so we expect a few of these to be Earth directed over the coming months. When aimed at Earth, they can reach the planet between one and three days later, and cause geomagnetic storms when they interact with the planet’s magnetic field. They can also amplify the northern and southern lights displays over the Earth’s poles.

They sun has been doing this for hundreds of millions of years and we’ve know about it since the the time of Galileo so why are we only just now getting concerned? Well. The biggest changers humanity has happened in the last 50 years. We are utterly dependent on telecommunications, the power grid, and oil. A big solar flare could stop any or all of these temporarily, and we can only imagine the problems that would bring.

The sun shines


For anyone currently alive and living in USA, mark August 21, 2017 on the calendar

I could never hope to put together words that describe the beauty and majesty of a solar eclipse. As the moon blocks out the sunlight, your skin crawls. For just a moment (2 mins 40sec) all the rest of world seems not to matter. Everything falls away and you finally really see the Sun. Everything falls away and you feel small, scared and entirely insignificant.

Now, If you live on mainland USA, consider how lucky you are. First, the moon and earth are the only two bodies in the solar system of exactly the right size and distance apart to create a total eclipse. Second, the moon is gradually moving away from the earth. Several tens of thousands of years ago, the moon was too close and so too large in the sky. Several tens of thousands of years from now it will be too far away and too small in the sky. Third, most solar eclipses occur over the sea or in places difficult to reach ( and expensive). But in 2017 all you have to do is pop a tent in your car and drive. Go see this event. It will stick in your mind for the rest of your life and you too will struggle for words to describe it.

The sun is doing the twist

The controversial of why the sun is so hot gets a revisit.

The issue is that the outer layers of the sun are hotter than the surface. There have been many theories to explain this and even quite a few papers claiming to have solved it. But the issue has never been cleared up satisfactorily. Now along comes a new piece of work. Solar tornadoes have the potential as conduits for transferring heat. These twisting bundles of magnetic field appear all over the sun and are big and frequent enough to possibly solve the issue. As to whether this clears or muddies the water, let’s wait and see.


Where does the solar system end?



It has to end somewhere. Scientifically the solar system has more than one end point. More correctly the sun has more than one end point. The sun gives off a constant steam of particles. At earth we are inside this stream, so even here we are still in the sun. The latest new from the voyager mission is has now reached a new boundary, 11 billion miles from earth. Eventually the stream of particles from the sun hits interstellar space and slows down. Other changes, in the magnetic field strength and direction should also happen. Voyager will keep going and going, so it will be become the first man made object to leave the Sun. The big question is, with only a few dozen years of battery left, will we know when it does so?