Details
| Original language | English |
|---|---|
| Title of host publication | Proceedings of Mid-term Symposium “3D Spatial Information Science – The Engine of Change” |
| Pages | 403-410 |
| Number of pages | 8 |
| Publication status | Published - 2018 |
| Event | ISPRS TC IV Mid-Term Symposium on 3D Spatial Information Science - The Engine of Change - Delft, Netherlands Duration: 1 Oct 2018 → 5 Oct 2018 |
Publication series
| Name | International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives |
|---|---|
| Publisher | International Society for Photogrammetry and Remote Sensing |
| Volume | XLII-4 |
| ISSN (Print) | 1682-1750 |
Abstract
In this work, we consider the exploitation of social media data in the context of Remote Sensing and Spatial Information Sciences. To this end, we explore a way of augmenting and integrating information represented by geo-located feature vectors into a system for the classification of satellite images. For that purpose, we present a quite general data fusion framework based on Convolutional Neural Network (CNN) and an initial examination of our approach on features from geo-located social media postings on the Twitter and Sentinel images. For this examination, we selected six simple Twitter features derived from the metadata, which we believe could contain information for the spatial context. We present initial experiments using geotagged Twitter data from Washington DC and Sentinel images showing this area. The goal of classification is to determine local climate zones (LCZ). First, we test whether our selected feature maps are not correlated with the LCZ classification at the geo-tag position. We apply a simple boost tree classifier on this data. The result turns out not to be a mere random classifier. Therefore, this data can be correlated with LCZ. To show the improvement by our method, we compare classification with and without the Twitter feature maps. In our experiments, we apply a standard pixel-based CNN classification of the Sentinel data and use it as a baseline model. After that, we expand the input augmenting additional Twitter feature maps within the CNN and assess the contribution of these additional features to the overall F1-score of the classification, which we determine from spatial cross-validation.
Keywords
- Classification, Data fusion, Deep learning, Satellite images, Social media mining
ASJC Scopus subject areas
- Computer Science(all)
- Information Systems
- Social Sciences(all)
- Geography, Planning and Development
Sustainable Development Goals
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
Proceedings of Mid-term Symposium “3D Spatial Information Science – The Engine of Change”. 2018. p. 403-410 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives; Vol. XLII-4).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Improved classification of satellite imagery using spatial feature maps extracted from social media
AU - Leichter, Artem
AU - Wittich, Dennis
AU - Rottensteiner, Franz
AU - Werner, Martin
AU - Sester, Monika
N1 - Funding information: This work was partially funded by the Federal Ministry of Education and Research, Germany (Bundesministerium für Bildung und Forschung, Förderkennzeichen 01IS17076). We gratefully acknowledge this support.
PY - 2018
Y1 - 2018
N2 - In this work, we consider the exploitation of social media data in the context of Remote Sensing and Spatial Information Sciences. To this end, we explore a way of augmenting and integrating information represented by geo-located feature vectors into a system for the classification of satellite images. For that purpose, we present a quite general data fusion framework based on Convolutional Neural Network (CNN) and an initial examination of our approach on features from geo-located social media postings on the Twitter and Sentinel images. For this examination, we selected six simple Twitter features derived from the metadata, which we believe could contain information for the spatial context. We present initial experiments using geotagged Twitter data from Washington DC and Sentinel images showing this area. The goal of classification is to determine local climate zones (LCZ). First, we test whether our selected feature maps are not correlated with the LCZ classification at the geo-tag position. We apply a simple boost tree classifier on this data. The result turns out not to be a mere random classifier. Therefore, this data can be correlated with LCZ. To show the improvement by our method, we compare classification with and without the Twitter feature maps. In our experiments, we apply a standard pixel-based CNN classification of the Sentinel data and use it as a baseline model. After that, we expand the input augmenting additional Twitter feature maps within the CNN and assess the contribution of these additional features to the overall F1-score of the classification, which we determine from spatial cross-validation.
AB - In this work, we consider the exploitation of social media data in the context of Remote Sensing and Spatial Information Sciences. To this end, we explore a way of augmenting and integrating information represented by geo-located feature vectors into a system for the classification of satellite images. For that purpose, we present a quite general data fusion framework based on Convolutional Neural Network (CNN) and an initial examination of our approach on features from geo-located social media postings on the Twitter and Sentinel images. For this examination, we selected six simple Twitter features derived from the metadata, which we believe could contain information for the spatial context. We present initial experiments using geotagged Twitter data from Washington DC and Sentinel images showing this area. The goal of classification is to determine local climate zones (LCZ). First, we test whether our selected feature maps are not correlated with the LCZ classification at the geo-tag position. We apply a simple boost tree classifier on this data. The result turns out not to be a mere random classifier. Therefore, this data can be correlated with LCZ. To show the improvement by our method, we compare classification with and without the Twitter feature maps. In our experiments, we apply a standard pixel-based CNN classification of the Sentinel data and use it as a baseline model. After that, we expand the input augmenting additional Twitter feature maps within the CNN and assess the contribution of these additional features to the overall F1-score of the classification, which we determine from spatial cross-validation.
KW - Classification
KW - Data fusion
KW - Deep learning
KW - Satellite images
KW - Social media mining
UR - http://www.scopus.com/inward/record.url?scp=85056156143&partnerID=8YFLogxK
U2 - 10.5194/isprs-archives-XLII-4-335-2018
DO - 10.5194/isprs-archives-XLII-4-335-2018
M3 - Conference contribution
AN - SCOPUS:85056156143
T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
SP - 403
EP - 410
BT - Proceedings of Mid-term Symposium “3D Spatial Information Science – The Engine of Change”
T2 - ISPRS TC IV Mid-Term Symposium on 3D Spatial Information Science - The Engine of Change
Y2 - 1 October 2018 through 5 October 2018
ER -