Have you read the top Pixalytics blogs of 2017?

World Cloud showing top 100 words from Pixalytics 2017 blogs

In our final blog of the year, we’re looking back at our most popular posts of the last twelve months. Have you read them all?

Of the top ten most read blogs, nine were actually written in previous years. These were:

You’ll notice that this list is dominated by our annual reviews of the number of satellites, and Earth observation satellites, orbiting the Earth. It often surprises us to see where these blogs are quoted and we’ve been included in articles on websites for Time Magazine, Fortune Magazine and the New Statesman to name a few!

So despite only being published in November this year coming in as the fourth most popular blog of the year was, unsurprisingly:

For posts published in 2017, the other nine most popular were:

2017 has been a really successful one for our website. The number of the views for the year is up by 75%, whilst the number of unique visitors has increased by 92%!

Whilst hard work, we do enjoy writing our weekly blog – although staring at a blank screen on a Wednesday morning without any idea of what we’ll publish a few hours later can be daunting!

We’re always delighted at meetings and conferences when people come up and say they read the blog. It’s nice to know that we’re read both within our community, as well as making a small contribution to informing and educating people outside the industry.

Thanks for reading this year, and we hope we can catch your eye again next year.

We’d like to wish everyone a Happy New Year, and a very successful 2018!

Big Data From Space

Last week I attended the 2017 Conference on Big Data from Space (BiDS’17) that was held in Toulouse, France. The conference was co-organised by the European Space Agency (ESA), the Joint Research Centre (JRC) of the European Commission (EC), and the European Union Satellite Centre (SatCen). It aimed to bring together people from multiple disciplines to stimulate the exploitation Earth Observation (EO) data collected in space.

The event started on Tuesday morning with keynotes from the various co-organising space organisations. Personally, I found the talk by Andreas Veispak, from the European Commission’s (EC) DG GROW department which is responsible for EU policy on the internal market, industry, entrepreneurship and SMEs, particularly interesting. Andreas has a key involvement in the Copernicus and Galileo programmes and described the Copernicus missions as the first building block for creating an ecosystem, which has positioned Europe as a global EO power through its “full, free and open” data policy.

The current Sentinel satellite missions will provide data continuity until at least 2035 with huge amounts of data generated, e.g., when all the Sentinel satellite missions are operational over 10 petabytes of data per year will be produced. Sentinel data has already been a huge success with current users exceeding what was expected by a factor of 10 or 20 and every product has been downloaded at least 10 times. Now, the key challenge is to support these users by providing useful information alongside the data.

The ESA presentation by Nicolaus Hanowski continued the user focus by highlighting that there are currently over 100 000 registered Copernicus data hub users. Nicolaus went on to describe that within ESA success is now being measured by use of the data for societal needs, e.g., the sustainable development goals, rather than just the production of scientific data. Therefore, one of the current aims is reduce the need for downloading by having a mutualised underpinning structure, i.e. the Copernicus Data and Information Access Services (DIAS) that will become operational in the second quarter of 2018, which will allow users to run their computer code on the data without the need for downloading. The hope is that this will allow users to focus on what they can do with the data, rather than worrying around storing it!

Charles Macmillan from JRC described their EO Data and Processing Platform (JEODPP) which is a front end based around the Jupyter Notebook that allows users to ask questions using visualisations and narrative text, instead of just though direct programming. He also noted that increasingly the data needed for policy and decision making is held by private organisations rather than government bodies.

The Tuesday afternoon was busy as I chaired the session on Information Generation at Scale. We had around 100 people who heard some great talks on varied subjects such as mass processing of Sentinel & Landsat data for mapping human settlements, 35 years of AVHRR data and large scale flood frequency maps using SAR data.

‘Application Of Earth Observation To A Ugandan Drought And Flood Mitigation Service’ poster

I presented a poster at the Wednesday evening session, titled “Application Of Earth Observation To A Ugandan Drought And Flood Mitigation Service”. We’re part of a consortium working on this project which is funded via the UK Space Agency’s International Partnership Programme. It’s focus is on providing underpinning infrastructure for the Ugandan government so that end users, such as farmers, can benefit from more timely and accurate information – delivered through a combination of EO, modelling and ground-based measurements.

It was interesting to hear Grega Milcinski from Sinergise discuss a similar approach to users from the lessons they learnt from building the Sentinel Hub. They separated the needs of science, business and end users. They’ve chosen not to target end users due to the challenges surrounding the localisation and customisation requirements of developing apps for end users around the world. Instead they’ve focussed on meeting the processing needs of scientific and business users to give them a solid foundation upon which they can then build end user applications. It was quite thought provoking to hear this, as we’re hoping to move towards targeting these end users in the near future!

There were some key technology themes that came of the presentations at the conference:

  • Jupyter notebooks were popular for frontend visualisation and data analytics, so users just need to know some basic python to handle large and complex datasets.
  • Making use of cloud computing using tools such as Docker and Apache Spark for running multiple instances of code with integrated parallel processing.
  • Raw data and processing on the fly: for both large datasets within browsers and by having the metadata stored so you can quickly query before committing to processing.
  • Analysis ready data in data cubes, i.e. the data has been processed to a level where remote sensing expertise isn’t so critical.

It was a great thought provoking conference. If you’d like to get more detail on what was presented then a book of extended abstracts is available here. The next event is planned for 19-21 February 2019 in Munich, Germany and I’d highly recommend it!

Living Planet Is Really Buzzing!

Living planet rotating global in the exhibition area, photo: S Lavender

Living planet rotating global in the exhibition area, photo: S Lavender

This week I’m at the 2016 European Space Agency’s Living Planet Symposium taking place in sunny Prague. I didn’t arrive until lunchtime on Monday and with the event already underway I hurried to the venue. First port of call was the European Association of Remote Sensing Companies (EARSC) stand as we’ve got copies of flyers and leaflets on their stand. Why not pop along and have look!

The current excitement and interest in Earth observation (EO) was obvious when I made my way towards the final sessions of the day. The Sentinel-2 and Landsat-8 synergy presentations were packed out, all seats taken and people were crowding the door to watch!

I started with the Thematic Exploitation Platforms session. For a long time the remote sensing community has wanted more data, and now we’re receiving it in ever larger quantities e.g., the current Copernicus missions are generating terabytes of data daily. With the storage requirements this generates there is a lot of interest in the use of online platforms to hold data, and then you upload your code to it, or use tools provided by the platform, rather than everyone trying to download their own individual copies. It was interesting to compare and contrast the approaches taken with hydrology, polar, coastal, forestry and urban EO data.

Tuesday was always going to be my busiest day of the Symposium as I was chairing two sessions and giving a presentation. I had an early start as the 0800 session on Coastal Zones I was co-chairing alongside Bob Brewin –a former PhD student of mine! It was great to see people presenting their results using Sentinel-2. The spatial resolution, 10m for the highest resolution wavebands, allows us to see the detail of suspended sediment resuspension events and the 705 nm waveband can be used for phytoplankton; but we’d still like an ocean colour sensor at this spatial resolution!

In the afternoon I headed into European Climate Data Records, where there was an interesting presentation on a long time-series AVHRR above-land aerosol dataset where the AVHRR data is being vicariously calibrated using the SeaWiFS ocean colour sensor. Great to see innovation within the industry where sensors launched one set of applications can be reused in others. One thing that was emphasised by presenters in both this session, and the Coastal Zone one earlier, was the need to reprocess datasets to create improved data records.

My last session of the day was on Virtual Research, where I was both co-chairing and presenting. It returned to the theme of handling large datasets, and the presentations focused on building resources that make using EO data easier. This ranged from bringing in-situ and EO data together by standardising the formatting and metadata of the in-situ data, through community datasets for algorithm performance evaluation, to data cubes that bring all the data needed to answer specific questions together into a three- (or higher) dimensional array that means you don’t spend all your time trying to read different datasets versus ask questions of them. My own presentation focused on our involvement with the ESA funded E-Collaboration for Earth Observation (E-CEO) project, which developed a collaborative platform  where challenges can be initiated and evaluated; allowing participants to upload their code and have it evaluated against a range of metrics. We’d run an example challenge focused on the comparison of atmospheric correction processors for ocean colour data that, once setup, could easily be rerun.

I’ve already realised that there too many interesting parallel sessions here, as I missed the ocean colour presentations which I’ve heard were great. The good news for me is that these sessions were recorded. So if you haven’t be able to make to Prague in person, or like me you are here but haven’t seen everything you wanted there are going to be selection of sessions to view on ESA’s site, for example, you can see the opening session here.

Not only do events like this gives you to a fantastic chance learn about what’s happening across the EO community, but they also give you the opportunity to catch up with old friends. I am looking forward to the rest of the week!

Want to know the top ten Pixalytics blogs of the year?

Artist's rendition of a satellite - paulfleet/123RF Stock Photo

Artist’s rendition of a satellite – paulfleet/123RF Stock Photo

Have you read all of our 2015 blogs? Did you miss a few weeks for a holiday? Whatever your answers, it turns out you may not have seen our most widely read blog last year.

As this is the final blog of the year we like to take a look back over the past fifty-two weeks and see which blog’s captured people’s attention, and conversely which did not!

So what have we discovered? Well, five of the top six most read blogs of 2015 were not actually written in 2015, but in 2014! This is a really positive thing for us, as it means our writing has a currency beyond the week/month/year in which it was written. The most widely read blogs in 2015, written in 2014, were in order:

  • How many Earth observation satellites are in space?
  • What do colours mean in satellite imagery?
  • How many satellites are orbiting the Earth?
  • Why understanding spatial resolution is important?
  • Remote sensing big data: possibilities and dangers

The remainder of the top ten were written in 2015, and in order were:

  • How many satellites are orbiting the Earth in 2015?
  • Mastering Landsat images in 5 simple steps!
  • Why counting animals from spaces isn’t as hard as you think?
  • Five Landsat quirks you should know
  • How many Earth observation satellites in orbit in 2015?

The eagled eyed amongst you will have noticed an interesting overlap between the two lists, namely the obvious interest in the number of satellites, and Earth observation satellites, orbiting the planet. I have a strong feeling a 2016 update will occur sometime next year!

We know counting the number of views of the blogs doesn’t give a true picture, as blogs issued earlier in the year are likely to have been read more than later ones. Therefore I’d like to give an honourable mention to three blogs written in November and December that still made it into the top 20, despite their limited time. These were:

  • Pixalytics is growing!
  • Practical Handbook of Remote Sensing
  • Sentinel-2 data released into the wild

This is our second year of weekly blog writing, and it has got a bit easier. We try wherever possible to have the blog written by Tuesday night, so it is ready to go out the next day. This has eliminated a lot of the pressure we had last year; arriving at the office on a Wednesday morning knowing we had a blog to issue in two hours and nothing written!

One thing we do ask ourselves each year, is whether all of this effort is worth it? I know if you read all the social media experts they will tell you it is vital to write a blog, but we think about whether our blog adds value to our business?

The answers this year came from:

  • Geo-Business 2015 and the 2015 UK Space Conference – We exhibited at both of these conferences and had a significant number of people come up to our stand and tell us that they read, and enjoyed our blog, which was great to hear.
  • Catalin, our new Erasmus student – If you read last week’s blog you’ll know that Catalin found Pixalytics by seeing a blog written by our summer Erasmus student, Selin.
  • Expert Authority – We know potential clients read our blog before developing a relationship with us, and it gives them a level of confidence in terms of Pixalytics being a company who knows its field and are up to date with what is happening.

We think the blog does add value to our business, and we intend to carry on next year.

We’d like to wish everyone a Happy New Year, and a very successful 2016!

Thanks for reading.

Remote Sensing and Agriculture in Italy

Poster from the Game Changing Technologies in Agriculture Event in Milan on 1st October 2015

Poster from the Game Changing Technologies in Agriculture Event in Milan on 1st October 2015

Last week I was in Italy talking all things remote sensing and agriculture. At the start of the week I was in Rome with the European Space Agency (ESA) discussing the Sentinel-2 performance, before catching the train north to Milan on the Wednesday evening for a series of UK Trade and Industry events focused on technology in the agricultural industry (AgriTech).

Thursday’s event was titled ‘Game Changing Technologies in Agriculture’, and was held in what looked like a large greenhouse in the grounds of Villa Necchi. We began the day with a welcome from the UK’s Ambassador to Italy, which was followed by talks from those working most directly in the agriculture industry. It was fascinating to hear some of the facts and see how much of a technological revolution has been occurring within this field. This is being driven by both the world population’s increased need for food – a 60% increase in demand by 2020 – and the corresponding need for businesses to increase their productivity. An overriding theme was the need to be more robust to, or better understand, the environment, including protection food production from both the weather and pests to reduce wastage.

After coffee we moved onto the provision of technological solutions, and there were a couple talks about how both drone and satellite remote sensing could benefit agriculture. My favourite other talks included the fitting of accelerometer collars on cows to collect data about their move movements more effectively, and the use of robot mechanical hands to perform repetitive tasks.

The afternoon expanded into synthetic biology, nanotechnology and technologies to reduce energy requirements or produce it more sustainably. Refrigeration is an important technology for the developing world, allowing a reduction in the current 40% post-harvest food wastage, but needs to be undertaken efficiently; the engines powering the refrigeration units on lorries produce much more pollution than the lorry engines themselves. This was followed by an interactive session where UK innovation centres had ‘stands’ that were used as discussions points on issues such as crops, horticulture, livestock, aquaculture, satellite technologies and big data. The day concluded with talks by Williams Advanced Engineering (associated with the Williams Formula One team) and IBM on how technologies are crossing from one sector to another.

On Friday, whilst it rained heavily in Milan, I spent the morning at the first Sainsbury’s Italian supplier conference. It was interesting to see how a large company is defining, and following, its strategies that include a focus on simplification; both for the supply chain and what the customer experiences. In the afternoon we had an escorted visit around the Milan Expo 2015. This is a six month exhibition which began in May and runs to the end of October and has the theme of Feeding the planet, energy for life’; it has exhibititors from over 140 countries and an exhibition area of 1.1 million square metres; although I didn’t explore all it! The UK exhibit was a beehive structure and wildflower meadow that was connected back to a real beehive in UK.

It was an interesting week and gave me lots of food for thought on how we can further develop the AgriTech services Pixalytics offers.

Remote Sensing Big Data: Possibilities and dangers

Remote sensing is an industry riding the crest of the big data wave. It offers great opportunities to those that can harness the power, but it’s also fraught with dangers. Big data is a blanket term used to describe datasets that are large and complex, due to the quantity of data, the speed at which new data becomes available or the variety of data. Remote sensing ticks all three of these boxes!

Sentinel-1 Netherlands

Sentinel-1 image of the coast of the Netherlands; courtesy of ESA

When I first started working with remote sensing, I approached the IT department to ask for 100 megabytes of disk space for my undergraduate project and was told nobody ever needs that much storage! Currently, the amount of Earth observation data available to the community is growing exponentially. To give you some examples, the recently launched Copernicus Sentinel 1-A satellite collects around 1.7 terabytes of data daily, the number of daily images collected by Landsat 8 has been increased by 18% this month and DigitalGlobe estimates it captures two petabytes of data each year. This quantity of data gives two key challenges; firstly, where to store it? Secondly, how do you know what data is valuable to enhance your decision-making?

It’s assumed the storage issue has been resolved by cloud computing, but there is a cost for getting the data to, and from, the cloud. An interesting recent study by the University of British Columbia discovered that over 80% of scientific data is lost within 20 years, mostly due to obsolete storage devices and email addresses. I have first-hand experiences of this. My PhD data was stored on hundreds of floppy disks and when I came to use them recently most didn’t work; fortunately I have a zip drive backup – although I still need to work out how to read Quattro Pro spreadsheets! I also have several Sun workstations with associated data on tapes which will only read from the machines they were written on; so how much of this data is accessible is debatable.

How often do you think about your old and archived data? Take a moment to consider how, and where, your critical data is stored. Is all of your data available and accessible? When was the last time the back-up procedures for your scientific or business data were tested? Does your IT department know which email addresses are critical for the receipt of satellite data?

The second challenge is knowing what data to use, particularly for people new to remote sensing. There is free data, paid for data, various satellites, various data types, various formats and the list can go on. The remote sensing community needs to help by providing more bridges between the data and the user community. The datasets available can offer huge benefits for business and science, but if people have to spend hours hunting round and trying to find the right image for them, they won’t stay users for long.

You can hire remote sensing companies, like us, who can offer impartial advice to help you select the right information. Pixalytics is striving to find more ways to make data more available, more accessible and more understandable. Remote sensing data belongs to everyone, and we need to support users to get it.