Evaluating the Quality of Authoritative Geospatial Datasets

General Directorate of Surveying is considered one of the most important sources of maps in Iraq. It produced digital maps for whole Iraq in the last six years. These maps are produced from different data sources with unknown accuracy; therefore, the quality of these maps needs to be assessed. The main aim of this study is to evaluate the positional accuracy of digital maps that produced from General Directorate of Surveying. Two different study areas were selected: ALRusafa and AL-Karkh in Baghdad / Iraq with an area of 172.826 and 135.106 square kilometers, respectively. Different statistical analyses were conducted to calculate the elements of positional accuracy assessment (mean μ, root mean square error RMSE, minimum and maximum errors). According to the obtained results, it can be stated that the maps of the General Directorate of Surveying can be used in reconnaissance or in works that require low or specified positional accuracy (eg. ±5m), and it cannot be used for applications need high accuracy (e.g. precise surveying).


INTRODUCTION
Maps provide important information about the resources of earth and about the distribution of these resources and help to identify their locations.Therefore, information from accurate maps should be extracted; otherwise, the results will be surprised and may be unacceptable.Many reasons lead to evaluating the map accuracy.First, a user may need to know how the extent of the map that has been produced or it is needed to enhance or increase the quality of a map by determining and correcting the errors sources.Second, it needs to compare between techniques, algorithms and analysts to know which is best.Accuracy evaluation can be qualitative or quantitative, expensive or inexpensive, quick or time consumption, has good and efficient designed or inefficient.Accuracy evaluation identifies the accuracy of map which are created from different datasets, thus, it is important and required, Congalton and Green, 2009.In this research, the positional accuracy is adopted which is considered the most important type of quantitative parameters.
Positional accuracy is the degree of matching of point coordinates taken from the map or imagery which desired to evaluate its accuracy with the coordinates of the same points taken from survey or other sources that considered more accurate.Positional accuracy can be referred to horizontal (in plane) or vertical (elevation) accuracy, Russell, 2009.Several sources of error can affect the positional accuracy of the map or referenced image, such as scanning errors, the inherent error in the original map that used to produce these maps, image processing errors, errors that occur during the collect of point samples and editing errors, Yanwei, 2005.
Several studies have been previously conducted to evaluate the positional accuracy of the various spatial data sources.For instance, Potter , 2008, addressed horizontal positional accuracy for Google Earth high-resolution imagery archive (A global images collection with roughly 2.5 meter resolution) using 436 control points distributed in 109 cities around the world.This was carried out by comparing the locations of points from Google Earth with its locations from the Landsat GeoCover which is an orthorectified product has absolute positional accuracy of less than 50 meters Root Mean Square Error (RMSE).In order to assess the accuracy of Google Earth, the locations of control points in Google Earth was compared to their corresponding locations in Landsat.It was concluded that this sample of the control points has an overall accuracy of 39.7m RMSE and the accuracy of these points was 24.1m RMSE in the more developed countries, which are more accurate than the control points in developed countries that have an accuracy of 44.4m RMSE.This means that the Google Earth high-resolution imagery is sufficient to use in analyzing the remote sensing results with medium resolution.
In another study, Koukoletsos et al , 2011, evaluated the positional accuracy of OpenStreetMap (OSM) data by comparing it with commercial data using a simplified version of the Increasing Buffer Method.The study showed that OSM can be relied upon as a source of data, but there are some limitations in its completeness in rural areas.Farah and Algarni, 2014, estimated the positional accuracy of Google Earth in Riyadh city by appointed nine stations in the campus of the King Saud University.They measured their coordinates using differential static-GPS technique.The study concluded that the error (RMSE) of Google Earth is up to (2.18m), and these results were similar to the results that obtained from other researches that conducted to evaluate the positional accuracy of Google Earth.In 2015, Pulighe et al, analyzed the horizontal accuracy of Google Earth images of Rom / Italy city for three different years: 2007, 2011 and 2013.They used (41) GPS points and (57) control points as an accurate source of data.Accuracy was measured by calculating the differences between the real coordinates and the coordinates taken from the images.The data were statistically analyzed using the laws of descriptive statistics and statistical graphics such as (histograms and Q-Q plot).The obtained results showed that the Google Earth very high resolution imagery of Rom has accuracy about 1m and it is possible to use these images to obtain precise measurements and produce large-scale maps.
This literary review showed how some of different spatial data sources are accurate in addition to the possibility of using these data in some surveying works such as taking measurements and producing maps in different scales depending on the accuracy of data.The main purpose of this study is to test the positional accuracy of the maps produced from the General Directorate of Surveying in Iraq for AL-Karkh and AL-Rusafa in Baghdad using different statistical measurements and graphs.

DATASETS
In the last six years ago, the General Directorate of Surveying in Iraq produced digital maps for whole Iraq with different detailed features.These maps are reliable as a source of spatial data for different surveying applications.However, these maps, similar to other geospatial data sources, need to ensure their quality to rely upon, especially for the applications that require high accuracy.Thus, in this study, the accuracy of the horizontal position of the maps produced from the General Directorate of Surveying was assessed.They were produced from several and different sources.These sources are: satellite images, aerial photos, field surveying and Military Survey Maps.These maps were produced using GIS software.These maps were referenced to WGS84 spheroid and projected into UTM.In order to assess the accuracy of the General Directorate of surveying maps, an aerial photo of 10 cm resolution that was referenced to the WGS84 spheroid was adopted as a reference data.These aerial photo maps cover the two a case study areas.

STUDY AREA
A photo map of Baghdad city was used to evaluate the accuracy of the maps of General Directorate of Surveying.Two parts of Baghdad were chosen: AL-Karkh and AL-Rusafa as two case studies.The study area in AL-Rusafa covers about 172.826 square kilometers, while the part taken from AL-Karkh covers about 135.106 square kilometers.Both of the study areas contain various features such as paved and non-paved roads, pedestrian roads, major roads, agricultural areas, residential areas, hospitals, train stations, railways and other features that have been reliable in the accuracy assessment.Fig. 1 and Fig. 2 show the case study areas that taken from the Baghdad city map.

THE SAMPLE OF POINTS
Horizontal positional accuracy evaluation was conducted by selecting 186 and 120 tested points from AL-Rusafa and AL-Karkh respectively.The coordinates of these points were extracted using ArcGIS10.2software program from corrected Baghdad photo map.The tested points were selected as sharp, clear and non-removable points such as road junctions, road endings and beginnings.Fig. 3 and Fig. 4 show examples of point locations in AL-Rusafa and Karkh.These points have been selected to be evenly distributed over the study area to represent all selected part.Horizontal coordinates of these points were measured from both the targeted data (General Directorate of Surveying maps) and the reference data (aerial image) and saved as tables.Horizontal coordinates of these points were saved as excel sheet in order to be imported in the MATLAB code.This code has been programmed using MATLAB programming language in order to calculate the elements of positional accuracy assessment.

EVALUATION POSITIONAL ACCURACY
Horizontal positional accuracy assessment was conducted on data from the General Directorate of Surveying / Iraq maps using descriptive statistics and some statistical tests such as t-test and ftest.The descriptive statistics laws were applied to determine statistical elements such as (mean µ, RMSE, minimum and maximum values).They can be considered as a metric accuracy measurement.The determination of spatial error in the elements requires calculating the differences between the coordinates of test points to assess their accuracy (coordinates from General Directorate of Surveying maps) and the reference coordinates (coordinates from aerial image).Equations used to calculate these elements are presented in the next section.In addition, the statistical tests that used in this study to calculate the errors convergence between the two case studies will be also presented and discussed.

Statistical Procedures
In this study, the horizontal positional accuracy of maps is assessed for two components of coordinates (Easting (E) and Northing (N)).This means that each of the statistical elements will be applied in the E-and N-components.The calculations of statistical elements will depend on computing the differences between the coordinates of the reference and target points (∆E, ∆N).
Eq. ( 1) can be used to calculate the arithmetic mean which is an expected value of error: The RMSE is the square root of the mean of the differences between the reference coordinates and the map coordinates and it can be calculated as follows: The same procedure has been applied to the y-coordinate of tested points.

Results of Descriptive Statistics
The points which were extracted from the General Directorate of Surveying maps for the parts AL-Karkh and AL-Rusafa have been statistically analyzed after excluding outlier points from them.The MINITAP software version 16.1 was utilized to analyze the data graphically using histogram and probability plots.The positional error values for points extracted from the parts of AL-Karkh and AL-Rusafa along Easting (E) and Northing (N) coordinates have been calculated by descriptive statistics laws as listed in Table 1.
It can be seen from Table 1. that the mean of errors for the differences in easting component (∆E) were 8.987m and 8.889m for AL-Rusafa and AL-Karkh, respectively, while the mean of errors in the northing component (∆N) were 13.010m and 11.480m for AL-Rusafa and AL-Karkh.The RMSEs were 12.1553m and 11.2108m for the easting components while they were 16.2001 and 13.9143 for the northing component for AL-Rusafa and AL-Karkh, respectively.It can be also noted that the resultants of the RMSEs (  ) were 20.2532m and 17.8686m for AL-Rusafa and AL-Karkh, respectively.This means that there are errors in the point positions in both easting and northing components.It can be also noted that the errors in the AL-Rusafa part larger than the errors in AL-Karkh part and this is clear from the minimum and maximum values.The maximum error in the easting components reach to about 40 m for AL-Rusafa and 27 m for AL-Karkh.While the maximum error in the northing components reach to 38 m and 30 m for AL-Rusafa and AL-Karkh, respectively.This means that the error values in point positions in AL-Rusafa part are larger than the differences in AL-Karkh part.

Histogram Analysis
Histogram is one of the statistical tools that graphically represent distribution of numerical data that grouped into frequency or percentage.Histograms give first indication of normally errors distribution.

AL-Rusafa Case Study
Histograms of AL-Rusafa/Baghdad show that the normal distribution curves do not match the easting and northing components very well.The histogram of easting component of AL-Rusafa Fig. 5 shows that the most errors ranging between (0-15) meters and the largest proportion of the points contains inaccuracies ranging between (0-5) meters.It can be also seen sharp peaks on the left of the histogram.The rest of the errors ranging between (20-40) meters with deviations on the right side of the histogram.Fig. 6 shows the histogram of errors in the northing component of AL-Rusafa.It is clear from this figure that the large proportion of the errors in the points ranging between 1-15 m.It can be also noted that there is a great peak on the left side.This means that there is a large proportion of point's deviate of about 5-8 m from their true positions.Then the proportion of the errors increases and reaches to about 40 m but in fewer rates of points.

AL-Karkh Case Study
It can be shown from the histograms of AL-Karh/Baghdad in Fig. 7 and Fig. 8 that the normal distribution curves do not converge along easting and northing coordinates very well.It is clear that the errors in the easting component is centered between 0 to 10 meters, then the error values increase but in less rate of points.The errors rise to more than 25 meters in different rate of points.The majority of the points contain inaccuracies ranging from 3 to 5 meters.The northing component errors are concentrated between 1 to 19 meters.Then the errors increase to more than 30 meters, while the proportion of these errors is increase and decrease.The majority of the points contain inaccuracies ranging from 1 to 5 meters.There are sharp peaks on the left of the histogram for both easting and northing components.

AL-Rusafa Case Study
The probability plot follows a normal distribution for both the easting and northing components but there are a number of points in the both ends take the long tail pattern Fig. 9 and Fig. 10.This means that it bends from the straight line and do not follow the pattern of rest of the points, and this is usually bending from the bottom to the bottom and from the top to the top.It can be noted that the long tail pattern begins after approximately 25 m in the easting component and after 32 m in the northing component.It can be also noted that the points that contain errors of 25 m and above in the easting component are more deviate from the straight line, which represent the mean of errors, because of increasing the errors in these points.It can be seen from these figures that the long tail pattern in the northing component is increasing to the top than in the easting component for both case studies.

AL-Karkh Case Study
The probability plot of AL-Karkh follows a normal distribution for both easting and northing components but there are number of points in the both ends take the long tail pattern Fig. 11 and Fig. 12.The long tail pattern begins after approximately 20 m in the easting component while it begins after 27 m in the northing component.It can be seen that the points to the right of easting component, which represent the points that contain errors more than 20 m, are deviated from the mean (straight line) more than the points in the northing component because of increasing the errors in these points which lead to this deviation.

The Horizontal Shift Analysis
The horizontal shift can be calculated using Eq. ( 5) which represents the deviation of point from its true position in both directions easting (E) and northing (N).The horizontal shift was computed for each point in the both case studies.The minimum horizontal shifts were 0.740 m and 1.073 m for AL-Rusafa and AL-Karkh respectively, while the maximum horizontal shift reaches to more than 50 m for AL-Rusafa and more than 30 m for AL-Karkh.The mean horizontal shift was 17.218 m and 15.893 m for AL-Rusafa and AL-Karkh respectively as illustrated in Table 2. (3) Where: ∆E :The difference between reference coordinates and tested coordinates in the east direction.∆N:The difference between reference coordinates and tested coordinates in the north direction.
The histogram of the horizontal shift was produced for both the case studies as shown in Fig. 13 and Fig. 14.The horizontal errors follow the normal distribution curve in the two case studies but do not much the normal distribution curve very well.It can be seen that errors in AL-Rusafa part centered between (1 to 30) m then it increases and reaches to more than 50 m, while in AL-Karkh part, the errors rise in some points and become lower in the other until they reach to more than 30 m. From these figures and Table 2, it can be concluded that the horizontal shift errors in AL-Rusafa part greater than errors of AL-Karkh part and it can be also seen that errors in AL-Karkh part are more normally distributed.

STATISTICAL TESTS
This section will discuss the T-test and F-test that used in this research.T-test was used to see how close the errors in the study areas to each other.F-test was applied because it is one of the ttest requirements.T-test one of the important statistical tests and the most frequently used in studies that designed to detect statistical differences between the means of two samples, Heiman, 2013.Types of this test are: one sample t-test and two samples t-test.To conduct this test, two hypotheses should be assumed: null hypothesis (H0) and alternative hypothesis (Ha).While the F-test represents the ratio between the variances of two populations, and it is one of the t-test requirements, Berenson and Levine, 1998.
In the current study, two sample of T-test was used to determine whether the errors between the two case studies were convergent or not.MINITAB software was used to conduct this test because it contains tools to facilitate the calculations.Before applying T-test, F-test should be applied firstly to compare the variance of the two datasets.Before starting these tests, it should be applying normality test, which was designed to find whether the data are normally distributed and following the normal distribution curve or not.

The Statistical Tests Results
In order to test the difference in error values between the two parts of AL-Karkh and AL-Rusafa, two Samples T-tests were adopted to detect whether the errors of AL-Karkh convergence to errors of AL-Rusafa or not.Hypotheses for this statistical test are the null hypothesis which states as follow: In this test, (95%) has been used as confidence level and the results have been discussed based on the (P) value.If the P value is more than (0.05) this means that the mean of the two compared parts are equal and if the P value is less than (0.05) this means that the mean of the two compared parts are not equal.
In this study, after applying T-test on the coordinates taken from Baghdad map for the two case studies (AL-Karkh and AL-Resafa), the results showed that these values of coordinates have P value larger than 0.05 for both easting and northing.This means that the discrepancies between the two case studies were close.The results also showed that the P value for the easting component is greater than p value for northing component, which means that errors in the north coordinates greater than errors in the easting coordinates.The results from these tests are shown in Table 3.
The results obtained from applying two sample t-test have been analyzed as a box plot, see Fig. 15 and Fig. 16.The box plot reflects the convergence of errors in the two case studies graphically.The horizontal axis represents the different in easting and northing components in the two case studies while the vertical axis represents the errors in the two case studies.The upper circle refers to the mean value, the lower circle represents the median value and the straight line represents the different in means of the two case studies.The difference between the mean of the two case studies has been taken.The low value of the mean means that the errors between the two case studies are converges.From box plot analysis, it is clear that the difference in mean values between the study areas was 0.098m in easting component and 1.530m in northing component.These results showed that the match is good with respect to the easting coordinates but it becomes less in the northing coordinates.

CONCLUSIONS
This study presented the spatial accuracy assessment of the digital maps that produced by General Directorate of Surveying Iraq.Two study areas: AL-Karkh and AL-Resafa in Baghdad were selected as two case studies and the point samples were extracted in order to use them in the assessment.Different statistical tests and analysis were applied to compute spatial accuracy assessment.MATLAB and MINITAB software were utilized to represent the errors graphically.
After the elements of positional accuracy assessment were calculated, the mean of errors were 8.987m and13.01m of the easting and northing components of AL-Rusafa respectively, while the errors were 8.889m and 11.480m of the easting and northing components of AL-Karkh respectively.The RMSEs were 12.1553m and 16.2001m for the easting and northing components of AL-Rusafa, while they were 11.2108m and 13.9143m for the easting and northing components of AL-Karkh.
The statistical tests (t-test and f-test) were used in order to see whether the errors in these map are converge or not.Therefore, t-test was applied on the data that taken from the two case studies (AL-Rusafa and AL-Karkh).The results showed that the errors in the two case studies are converge.
From these results it can be shown that these maps have limited positional accuracy and they cannot be used in the precise surveying works (of accuracy, for example, ±0.5 m).While it can be used in the not precise surveying works (with accuracy, for example, ±2m) such as reconnaissance or academic studies.
mean   : reference coordinate (photo map coordinate)   : map coordinate (General Directorate of Surveying data) N: total number of tested points 2) RMSE: root mean square error   : reference coordinate (photo map coordinate) of east component   : map coordinate (General Directorate of Surveying data) of east component N: total number of tested points of AL-Resafa part µ2: mean of AL-Karkh part While the alternative hypothesis (H1) states (H1: µ1≠ µ2).

Figure 2 .
Figure 2. The study area of AL-Karkh (General Directorate of Surveying).

Figure 3 .
Figure 3. Point 1 in AL-Rusafa part: it is an intersection of two roads.

Figure 4 .
Figure 4. Point 23 in AL-Karkh part: it is an intersection of road and bridge.

Figure 5 .Figure 6 .
Figure 5. Histogram of the errors in Easting coordinates in AL-Rusafa map.

Figure 7 .Figure 8 .
Figure 7. Histogram of the errors in Easting coordinates in AL-Karkh map.

Figure 9 .
Figure 9.The probability plot of Easting coordinates in AL-Rusafa map.

Figure 10 .
Figure 10.The probability plot of Northing coordinates in ALRusafa map.

Table 1 .
The results of descriptive statistics of tested datasets.

Table 2 .
Horizontal shift in the two case studies.

Table 3 .
Results of statistical tests.