Positional Accuracy Assessment for Updating Authoritative Geospatial Datasets Based on Open Source Data and Remotely Sensed Images

O penStreetMap (OSM) represents the most common example of online volunteered mapping applications. Most of these platforms are open source spatial data collected by non-experts volunteers using different data collection methods. OSM project aims to provide a free digital map for all the world. The heterogeneity in data collection methods made OSM project databases accuracy is unreliable and must be dealt with caution for any engineering application. This study aims to assess the horizontal positional accuracy of three spatial data sources are OSM road network database, high-resolution Satellite Image (SI), and high-resolution Aerial Photo (AP) of Baghdad city with respect to an analogue formal road network dataset obtained from the Mayoralty of Baghdad (MB). The methodology of, U.S. National Standard Spatial Data Accuracy (NSSDA) was applied to measure the degree of agreement between each data source and the formal dataset (MB) in terms of horizontal positional accuracy by computing RMSE and NSSDA values. The study concluded that each of the three data sources does not agree with the MB dataset in both study sites AL-Aadhamiyah and AL-Kadhumiyah in terms of positional


INTRODUCTION
The latest developments of Web2 techniques over the last three decades have enabled users to create, share, download, embed, and edit their own generated content to the related Web2 platforms, such as Facebook, YouTube, Flicker, etc, )AL-Bakri, 2015 and O'Reilly, 2007(. In terms of the geospatial information, users can freely share and download their local spatial information to the open-access Web2 mapping projects such as OSM project, Google Maps, Wikimapia, Google Earth, Yahoo! Imagery, and others (Haklay, 2008). All of these geospatial platforms that are considered an open-licensed source data were grouped under the term Volunteered Geographical Information (VGI) by Mike Goodchild in 2007(Haklay, 2008. Online geographic information usually were creating by users using a set of techniques and tools that fall outside the realm of traditional Geographic Information Systems (GIS) (Turner, 2006). Moreover, the latest advancements in geospatial data collection methods have helped the nonexperts users to pick up their spatial data rapidly and easily using their portable GPS devices. Therefore, (Turner, 2006), defined this technology as 'Neogeography', which means geography without geographers. Most of the available collaborative geographical data services on the internet are creating and sharing by volunteer contributors, who have little experiences in surveying works, (Haklay, 2010). Therefore, these data were defined as a sort of User Generated Content (UGC) (Antoniou, 2010). The different geospatial data collection methods and unequal experience levels of users have made the accuracy of VGI is questionable and unreliable and need to be examined before using them for any engineering application. Therefore, the related researchers started to investigate the quality of those online freely available geographical data services such as OSM project. (Zielstra1 and Zipf, 2010) compared the positional accuracy between OSM datasets and Tele Atlas datasets in Germany in terms of positional accuracy and completeness. The Tele Atlas data is usually measured using the laser scanner and GPS techniques, therefore, the positional horizontal accuracy of these data can reach up to 1 meter. The results showed that the contributions made by the volunteers in Germany were very effective, so the OSM project is expected to outperform the Tele Atlas data in the near future. Moreover, the results showed that the bigger cities have higher completeness than the medium and small-sized cities. (Haklay, 2010) assessed the completeness and the positional accuracy of OSM road data in five selected areas in London /UK by comparing OSM datasets with an analog dataset from Ordnance Survey (OS) Master Map. The estimated positional horizontal accuracies in each area were 6. 77, 8.33, 6.04, 3.17, and 4.83m. This means that the positional horizontal accuracy of OSM data differs between the different regions in London, this is due to the digitization method, data collection skills, and the sabra of the contributor who conducted the work. (AL-Bakri and Fairbairn, Journal of Engineering Volume 26 February 2020 Number 2 72 2010) tested the positional accuracy and the shape quality of OSM data in the UK by comparing them with OS data and an accurate Field Survey (FS) dataset. The study concluded that the positional discrepancies between the OSM dataset and each of the two other datasets were very large. Also, the shape quality evaluation which was measured by using the invariant moment method indicated that the OSM dataset does not conflict with each of the two other datasets OS and FS in each of the two study sites, the urban site, and the rural site. The outcomes showed that the (TT) and OSM datasets have high positional accuracy and therefore they can be used to produce maps with medium and small scales. (Goudarzi, 2017) assessed the Google Earth horizontal positional accuracy in Canada, Montreal using an accurate GPS dataset. The findings showed that the positional accuracy was about 0.1 m in the south and about 2.7 m in the north. The study discussed the displacement problem which can occur due to the topography of the earth and the angle of view of the imaging satellite. (Zhang and Malczewski, 2018) assessed OSM street network data quality in terms of positional accuracy and completeness accuracy by comparing these data with a dataset from the national road network database in Canada, the Digital Mapping Technologies Inc. (DMTI) which published on Sept. 1, 2015. DMTI has a positional accuracy ranges from 0.6m (urban) to 30m (rural). The study concluded that the positional horizontal accuracy of OSM data was 25m with respect to DMTI. Moreover, the study showed that the urban areas received more contributions than rural areas. The diversity in the accuracy of data acquisition methods and the degree of experience of the keen volunteer's contributors to OSM project, make the accuracy of OSM project datasets is unreliable and questionable. In this research, the accuracy of the horizontal position has been assessed for three different spatial data sources for Baghdad city, OSM project road network data, Satellite Image (SI) with 0.50m resolution, and Aerial Photo (AP) with 0.10m resolution. The assessment process was carried out by comparing these three different datasets with an analogue authoritative geospatial dataset obtained from the official road network mapping agency of Baghdad city, the Mayoralty of Baghdad (MB). Different techniques and procedures were followed and applied to evaluate OSM dataset quality as well as the two raster datasets to determine the usability of each data source for updating the road network databases in Baghdad city for serving the collaborative mapping between the formal and informal datasets.

Case Study Description and Data Collection
Two study sites were selected in Baghdad city, the first study site is a part from AL-Aadhamiyah town which is in AL-Rusafa side and the second study site is a part from AL-Kadhumiyah town which is in AL-Karkh side as shown in the Fig. 1,a and Fig. 1,b. In this study, the areas of the two used parts are 17km² and 12km² of AL-Aadhamiyah and AL-Kadhumiyah, respectively. Each of these two study sites contains official and private buildings, residential areas, agricultural areas, main roads, highway roads, secondary roads, streets, pedestrian routs, parks, green lands, markets, and many other land use landscapes. The proportion of agricultural and wet areas in the study site of AL-Kadhumiyah town is larger than it in the study site of AL-Aadhamiyah town. These areas were extracted from the assessment process due to the absence of clear and systematic road intersection points in such these areas. Each of the two study sites is considered an important commercial and religious center in Baghdad city and attracts a lot of visitors for religious tourism and shopping in addition to their an indigenous population density. So the road network needs constant maintenance and development in order to avoid any problem in terms of congestion.
Four different road network data sources of the two selected study sites in Baghdad city were used to carry out this research, three of them are informal data sources, while the fourth source is a formal data source. The three informal datasets are the open-source OSM road network shapefile, Aerial Photo (AP) with spatial resolution of 0.10m and a Satellite Image (SI) with a spatial resolution of 0.50 m. To achieve the aims of this research, similar boundaries were taken for all datasets to ensure high level of assessment positional accuracy. For conducting the assessment, these three evaluated points datasets AP, SI, and OSM were compared with an analogue authoritative dataset as reference dataset obtained from Mayoralty of Baghdad (MB), the official road network mapping agency in Baghdad city. MB roads network dataset was traced from QuiqBird satellite images with resolution of 0.60 m using GIS software techniques which are referenced to the spheroid WGS84 and the projection UTM zone 38N. By contrast, most of OSM road data for Iraq are mapped by tracing the information from different resolution satellite images such as Yahoo! Satellite images, LANDSAT images, Esri satellite images, and more others (AL-Bakri, 2015). OSM data has many thematic layers which were created by Frederik Ramm on his website Geofabric using OSM tags. The OSM project data are available as free online source and can be downloaded with Esri shapefile format from Geofabrik website (http://geofabrik.de.com/). Geofabrik website is an open-source application was created by Frederik Ramm to process the contributed OSM datasets by the volunteers to provide a free digital map of all the world. OSM digital maps consist of many thematic layers including buildings, land use, rivers, roads, railways, waterways, nature, points, places, etc (Haklay, 2008). OSM data file has been downloaded for all the parts of Iraq with a file size of 35.8 MegaByte and the latest date modified was 17/4/2018. The downloaded file consists of a number of shapefiles, which are compatible with GIS package. Although, the OSM project data has many thematic layers, however, only road network layer was selected and displayed on GIS interface to conduct the assessment process as shown in Fig. 2. In terms of the two raster data sources, one georeferenced scene of each of the satellite image and the aerial photo was used to cover the two study sites. The georeferencing process was performed by ArcMap10.4.1 software using (7) ground control points, which were observed using PPK GPS technique around the two study areas. Fig. 3 illustrates the four different data sources in georeferenced vision in the ArcMap interface, where the red color shapefile represents MB road database while the blue color shapefile represents OSM road database. To conduct this study OSM road network shapefile which is in blue was extracted only for the selected two study areas, as shown in Fig. 3. ArcMap 10.4.1 software was used to extract the coordinates of the selected road intersection point datasets from the Aerial Photo (AP), the Satellite Image (SI), OSM shapefile, and MB shapefile for each study area. Moreover, Fig. 3 illustrates ISBA CORS point location with respect to the 7 GCPs locations.

Digitizing Intersection Point Datasets
The four intersection point datasets, MB, SI, AP, and OSM were digitized and saved in the projected reference coordinate system WGS48/UTM zone 38N using ArcMap10.4.1 software. The number of points, which were selected for each case study was, 1018 points in case of study one, AL-Aadhamiyah, while 377 points in case study two, AL-Kadhumiyah. Using large sample size is very important to conduct this study due to using too small sample size may lead to failure in detecting the differences between the two compared datasets that may be large and important, Felicien, 2017. Fig. 4 displays a sample of the digitized intersection points on the different four data sources with same IDs.

Horizontal Positional Accuracy Analysis
After digitizing the selected intersection points, the coordinates were exported in Excel format in order to apply the statistical tests. The Easting and the Northing discrepancies (ΔE, ΔN) between the MB dataset point coordinates and the corresponding coordinates in each of the three datasets AP, SI, and OSM were calculated using the following equations: Three Easting discrepancies datasets and three Northing discrepancies datasets for each case study were computed and exported in excel format for statistical analysis.
The horizontal discrepancies datasets were analyzed using the statistical analysis software Minitab 17 by finding the descriptive statistics for each dataset in order to describe the distribution pattern of the discrepancies values around the mean value. The findings indicated that the values of means and medians were close to each other for each dataset; this gives a good indication that all the datasets of horizontal discrepancies are normally distributed. Several standards have been designed and established to assess the positional accuracy of spatial data. All these standards were helpful but did not consider the digital geospatial data that can be easily manipulated and reproduced in varying scales and formats. Therefore, a new standard has been designed by the U. S. Federal Geographic Data Committee (FGDC) in 1996 as a comprehensive standard. This standard was known as the U. S. National Standard Spatial Data Accuracy (NSSDA) (FGDS, 1996). NSSDA allows users to investigate the positional accuracy of digital datasets by comparing them with spatial datasets have higher accuracy. This standard assesses the positional horizontal accuracy of geographical datasets by computing the Root Mean Square Error and NSSDA accuracy using confidence interval of 95% and significant level of 5% of these datasets (Congalton and Green, 2009). The strategy of NSSDA uses the following algorithms in order to assess the horizontal positional accuracy of geospatial datasets (Greenwalt, 1962).

Horizontal Shift Analyzing
The horizontal shifts (Euclidean Distances) between the point locations of formal MB dataset and the corresponding point locations that extracted from each of the three informal data sources Aerial Photo with resolution of 0.10m (AP), Satellite Image with resolution of 0.50m (SI), and OSM road network shapefile (OSM) were calculated by applying the following equation: Where: ΔE: the Easting difference between MB dataset points and the corresponding points of each of the other three datasets.
ΔN: the Northing difference between MB dataset points and the corresponding points of each of the other three datasets.

Descriptive Statistics Method
Descriptive statistics method was performed using Minitab 17 statistical software to analyze the horizontal positional shift datasets in each case study. The results are illustrated in Table 3 and Table 4 for AL-Aadhamiyah and AL-Kadhumiyah, respectively.     Table 4 show that the mean values of each dataset are within the 95% CI and the mean and median values are approximately close to each other for each case study. This is a remarkable indication that the datasets are normally distributed. Table 3 and Table 4 show that the dispersion items which are Std. Deviation, Maximum Shift, and Minimum Shift for each dataset are close to each other in each case study. This indicates that the three datasets flow the same dispersion pattern.

One-Sample T-Test Method
One-sample t-test is a hypothesis test used often to compare the means of two or more homogenous and normal distributed datasets, the first dataset is called the population dataset while the second dataset is called the tested dataset (Black, 2011). One-sample t-test assumes two hypotheses to compare the means, the first is the null hypothesis, which states ( 0 : ̅ = µ), the second hypothesis is the alternative hypothesis, which states (Ha: ̅ ≠ µ). Where (µ) is the mean of population and ̅ is the estimated mean value of the tested dataset. In this study, the population dataset is represented by the Mayoralty of Baghdad (MB) dataset. While the tested datasets are the three positional horizontal shift datasets MB-AP, MB-SI, and MB-OSM. Minitab 17 statistical software has been used to perform one-sample t-test (two-tailed test) for each of the three positional horizontal shift datasets MB-AP, MB-SI, and MB-OSM for each case study in order to compare the mean ( ̅ ) of each tested dataset and the mean of population (µ). and Table 6 summarize the results of one-sample t-test of the two study sites AL-Aadhamiyah and AL-Kadhumiyah, respectively.

Table 5. One-Sample T-Test Results for Horizontal Shift Datasets in AL-Aadhamiyah Case
Study.

OSM Data Analysis
The two-sample t-test is a hypothesis test that often used to compare the arithmetic means of two or more normally distributed homogeneous datasets. Therefore, this test was performed in this study to investigate the mean difference between the two MB-OSM horizontal shift datasets in the two study sites in order to assess the positional horizontal accuracy of OSM data in different study areas in Baghdad city. Two-sample t-test assumes that the arithmetic means values of the two tested datasets are equal in its null hypothesis (H₀ : µ₁ -µ₂ =0) while in its alternative hypothesis, it is assumed that the two means are not equal (Hₐ: µ₁ -µ₂ ≠0) (Black, 2011). In this study, µ₁ and µ₂ represent the mean values of MB-OSM horizontal shift datasets in AL-Aadhamiyah town and AL-Kadhumiyah town, respectively. The results of the two-sample t-test are summarized in Table 7.  Table 2 shows that the NSSDA values are 10.820 m, 11.359m, and 12.174m for MB-AP, MB-SI, and MB-OSM datasets, respectively. This indicates that 95% from the points in each of these three datasets have an error equal to or smaller than 10.820m, 11.359m, and 12.174m, respectively. While 5% of the points have an error value larger than 10.820m, 11.359m, and 12.174m, respectively. Whereas the total RMSE values were 6.359m, 6.625m, and 7.031m for the datasets MB-AP, MB-SI, and MB-OSM, respectively. The relatively large values of TOTAL RMSE and NSSDA accuracy indicate that the positional horizontal accuracy of each of the three tested datasets AP, SI, and OSM does not agree with the positional horizontal accuracy of the authoritative MB dataset. The most important reason for this disagreement may be the inaccurate image georeferencing process. It is clear from Table 1 and Table 2 that the TOTAL RMSE and NSSDA accuracy values of OSM dataset are larger than those of the other two datasets AP and SI. The most important reason in this large mismatching between MB dataset and OSM dataset is the diversity in data acquisition methods of OSM project by inexpert volunteers either by field surveying works such as real-time GPS trucks or by tracing the spatial information from the different resolution satellite images and aerial photos. Moreover, most of OSM datasets in terms of Iraq are traced from visually georeferenced satellite and aerial images without using GCPs (AL-Bakri and Fairbrain, 2013). Moreover, Table 1 and Table 2 indicate that the total RMSE and NSSDA values of the two datasets MB-AP and MB-SI are approximately close to each other in each case study. This indicates that the positional horizontal accuracy of these two datasets is approximately close to each other in both study areas. The most important reason of this approximated geometric matching between the two raster datasets AP and SI is conducting an accurate georeferencing process of these two rasters using an adequate number of GCPs. Table 5 shows that the P-value for each dataset is 0.000 and the t-values are 37.490, 37.360, and 36.780 for MB-AP, MB-SI, and MB-OSM datasets, respectively, in AL-Aadhamiyah case study. Table 6 shows that the P-value for each dataset is 0.000, while the t-values are 29.677, 30.899, and 31.782 for MB-AP, MB-SI, and MB-OSM datasets, respectively, in AL-Kadhumiyah case study. These findings are significant indications to reject the null hypothesis and accept the alternative hypothesis. Moreover, the values of Mean diff., are relatively considered large and do not allow matching between MB dataset and each of the other datasets AP, SI, and OSM. Table 7 shows that the estimated difference between means is (1.202). The 95% confidence interval for difference are (1.617, 0.787). The confidence interval is a range of likely values for the difference. Since, the true value of this difference is unknown, the confidence interval helps to guess its value based on the sample data. The difference between sample means provides an estimate of (µ1-µ2), and the samples' standard deviations are used to determine how the estimate of difference might be far off (Heiman, 2013). Moreover, the calculated t-value is 5.69, and the p-value is 0.000 for degree of freedom 742 and t-critical (tabulated) 1.962. This indicates that the t-value is not within the acceptance region and the p-value is less than 0.05; therefore, the null hypothesis must be rejected, this indicates that the difference mean is not equal to zero. This leads that the horizontal positional accuracy of the OSM road data is not similar in the two case studies AL-Aadhamiyah and AL-Kadhumiyah due to the difference mean which is estimated by 1.202m 9. CONCLUSIONS 1-After investigating the normal distribution of the horizontal shift datasets, which were computed between the formal dataset (MB) and each of the other used three datasets AP, SI, and OSM, NSSDA methodology was applied in order to measure the horizontal positional accuracy for each of the three datasets. The study concluded that each of the three tested datasets does not match MB dataset in both selected study areas in Baghdad city. Therefore, the study concluded that the geometric matching between such these large scale datasets is impossible and that the three informal data sources are not appropriate for updating the road network database in Baghdad city.

2-
For further investigating one-sample t-test have been performed to measure the horizontal positional accuracy for each data source. The resulted t-values from one-sample t-test for the three tested datasets are very closed to each other in each case study, as shown in Table 3 and Table 4. The closed calculated t-values to each other indicate that the horizontal positional accuracies of the three tested datasets are approximately similar. However, the relatively large mean difference values between MB dataset and the three tested datasets AP, SI, and OSM indicate that these three datasets are very far off from the authoritative MB dataset in terms of horizontal positional accuracy. Therefore, this study concludes that the three tested spatial data sources cannot be appropriate for updating road network databases in Baghdad city.

3-
The results of two-sample t-test which is performed to determine the degree of geometric matching between the two OSM datasets in the two selected study areas in Baghdad city led to reject the null hypothesis and accept the alternative hypothesis. Moreover, the study concluded from performing the two-sample t-test that the horizontal positional accuracy of OSM road network datasets is different in the two selected study areas in Baghdad city by estimated mean difference of about 1.202m. The major reasons for this difference are the varying accuracies of the different data acquisition methods by un-experts volunteers as well as the resulted errors from the transformation processes between the different coordinate reference systems.
10. RECOMMENDATIONS 1-Investigating the horizontal positional accuracy of OSM databases in other study areas throughout Iraq to determine the usability (fitness of use) of these databases for updating the authoritative road network databases in each municipality for serving the collaborative mapping.
2-For future works, it is recommended to compare the OSM road network data and remotely sensed road network data with an accurate road intersection point datasets obtained from a field survey using an accurate GPS positioning techniques. This accurate comparison can accurately assess the free available OSM data and remotely sensed images and photos. This can serve the collaborative mapping between the official agencies and the promising OSM project to save costs, efforts, and consuming time of the traditional mapping methods.
3-For future works, it is recommended to evaluate the positional accuracy of other road networks databases provided by other VGI services such as Google Maps, Wikimapia, ESRI road network databases, Google Earth Maps, etc. 4-For future works, it is recommended to evaluate the usability of other types of remotely sensed images that are freely available on the internet such as Google Earth images, Esri satellite imagery, Yahoo! Aerial imagery, etc and addressing the usability of each source for serving the collaborative mapping.