Big Data – Captain's Log : The Cape Town Diaries https://www.mattiavaccari.net/wp (Mostly) Astrophysical Ramblings from the Mother City & Beyond Tue, 21 Jul 2020 10:47:21 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 https://i0.wp.com/www.mattiavaccari.net/wp/wp-content/uploads/2015/09/cropped-mattia_avatar_5_square.jpg?fit=32%2C32&ssl=1 Big Data – Captain's Log : The Cape Town Diaries https://www.mattiavaccari.net/wp 32 32 66042253 (Data) Size Matters! https://www.mattiavaccari.net/wp/blog/2014/11/14/data-size-matters/ https://www.mattiavaccari.net/wp/blog/2014/11/14/data-size-matters/#respond Fri, 14 Nov 2014 09:27:46 +0000 http://www.mattiavaccari.net/wp/?p=105 Continue reading ]]> These days I’m attending a conference in Frascati, on the beautiful hills surrounding Rome, about large space satellite datasets and their exploitation. Most of the sessions so far have been about new data processing infrastructures and technologies and Earth Observations, but this morning’s session (opened by the excellent keynote by Naples’ Peppe Longo of DAME fame) is about Big Data in Astronomy, and some of the most ambitious upcoming astronomical ground-based projects such as LSST and SKA have also sneaked into the program. Throughout, much emphasis is being put on bridging the gap between the software engineering layer required to store the data and the actual scientific exploitation, including customizing Data Mining and Machine Learning techniques to Astronomy. And for those who were wondering: at a nominal data rate of 1 Exabyte / day (that’s a million Terabyte, or a billion Gigabyte, per day), the Square Kilometre Array perhaps unsurprisingly leads the pack;-)

]]>
https://www.mattiavaccari.net/wp/blog/2014/11/14/data-size-matters/feed/ 0 105