In the book The Clustering of America published 3 decades ago, Michael Weiss described the methodology in the geo-demographic analysis and neighborhood clustering.

In Hong Kong, the population census provides rich dataset for geo-demographic analysis, this is an attempt to analyze and dataset and group different attributes into clusters, in order to provide insights to further cluster the neighborhood.

The following dataset is downloaded from CSDI portal.

2021 Population Census (Statistics and Boundaries of District Council Constituency Areas)

Python script for feature selection is developed on the dcca dataset and following is the result dendrogram.

# Standardize features
scaler = StandardScaler()
df_scaled = scaler.fit_transform(df_features)
df_scaled = pd.DataFrame(df_scaled ,columns=df_features.columns)

# Transpose so that each row is a feature
df_transposed = df_scaled.transpose()

# Perform hierarchical clustering on features
linked = linkage(df_transposed , method='ward', metric='euclidean')

# Create a dendrogram to visualize the feature clustering
plt.figure(figsize=(120, 60)) 
dendrogram(linked ,
            orientation='top', 
            labels=df_transposed.index,
            distance_sort='descending', 
            show_leaf_counts=True, leaf_font_size=10)

Following are the result showing the cluster and sub-cluster by analyzing the above dataset.

Thanks.

Inspired by the well-being design concepts proposed by the Housing Authority last year, Geospatial HK developed a web application call Well-being Map, where 12 concepts are calculated base on the open source POI data from CSDI portal and Open Street Map (OSM).

Website: https://wellbeingmap.org/

e.g. The concept 活躍好動 is calculated by the following formular with the counts of different POI within each 1 Hectare hexagon covering Hong Kong.

4 * csdi_cycsite + 1 * osm_drinking_water + 2 * osm_picnic_site + 3 * osm_bicycle_shop + 1 * osm_camp_site + 4 * osm_sports_centre + 3 * osm_sports_shop + 3 * osm_bicycle_rental + 3 * osm_swimming_pool + 2 * osm_stadium + 6 * csdi_hiking_trails

It assumed that the citizen preferring this concept will also prefer living in the area with higher score of this concept.

The most interesting thing for this application is users can overlay different concepts with their preferred weight, and the map engine will calculate score of each hexagon base on the combined weight of different concepts in it. It mimics the raster overlay analysis in GIS software. (https://desktop.arcgis.com/en/arcmap/latest/analyze/commonly-used-tools/overlay-analysis.htm)

Through this process users can understand their real needs, and locate the areas in the city that are suitable to them.

E.g. If user prefers more on food (restaurant and market), he or she can put more emphasize on 講飲講食 concept, following by the natural environment concept (綠色自然), with some emphasize on 活躍好動, 文靜休閒, 出行便捷. And then the application will display the map which meet his or her personal needs.

The advanced version is under development, where users can define their own concepts with the formula they prefer, and more POI data set will be provided later. There will be also attempt to match the properties in the area calculated based on their personal needs, and with the highest CP value for them.

Thanks.

Project website: https://access.hk

This is a preliminary attempt of Geospatial HK to map the wheelchair accessibility hotspots in the city. Objective of this map is to pinpoint the area that is not optimally designed for wheelchair users or area that has natural limitation that is unfavourable to the wheelchair users. This study uses the 3D Pedestrian Network data set from CSDI portal.

Method

We define hexagon grid over the city, each grid is 1-Hectare large.

For each hexagon grid, we set its center as the starting point and searching for the shortest path reaching surrounding hexagon grid within a 400m radius. For each start end pairs there are 2 paths searched, one using the general pedestrian path, and the other one considering the wheelchair obstacles indicated in the 3DPN data set.

Then, we will calculate the barrier ratio which is

length of wheelchair path / length of general pedestrian path

For all end points of a starting point, and highest ratio will be the index of that starting point. The hexagon grid will be displayed in red if the index exceed 1.5, the higher the index and deeper the color of the grid.

When click on the hexagon grid, detail routes will be displayed, the blue one is general pedestrian path, the red one is the wheelchair access path, it has to take longer way because there is wheelchair obstacle such as staircase in between the general path.

We found that some wheelchair path is unreasonably longer than the general path, one of the reason is the incomplete accessible path data in the 3DPN data set, especially those semi-open public access allowing wheelchair to pass through. Beware of that this is just a preliminary study at the moment and for reference only and Geospatial HK bares no obligations on the correctness and completeness of the data set. We will improve the qualify of the map over time.

One of the future enhancement is to add the verification mechanism or site visit /Google Street Map cross checking capability so that public can verify the hotspots on the map. We also want to work with the corresponding stakeholders in the society to extend the usability of this map.

You can download a high resolution printable version here.

Thanks.

Geospatial HK developed a small tool for users to create custom 3x3 traffic camera dashboard using HK Gov open traffic data. The steps are simple.

1. open a web browser and visit dashboard.hk, and click on the link "Custom 3x3" on the left menu.

2. This is the tool to compose the custom dashboard.

3. For example, if you want to display the Time and Weather information in the middle grid of the 3x3 dashboard i.e. Grid 5. Select the radio button 5, and select "Time & Weather".

4. Click the Green Down Arrow button to assign the content to the grid.

5. The content will be previewed in the 3x3 grid at the bottom. If you want to undo, you can click the Unassign button in the preview panel.

6. Now, add a traffic cam to Grid 1, select grid 1, and select Traffic Cam radio button.

7. In the map, click on the camera icon you want to add. The selection list below will be populated, optionally, you can manually select the camera from the list.

8. Click the assign button.

9. The camera snapshot will be previewed in the selected grid.

10. Similarly, let's assign other grids. After that, click the Launch button.

11. A new window will be opened with the selected contents inside. The camera photos will be refreshed every 30 seconds from the source.

12. It is suggested to bookmark the link for future use.

Thanks.

We have developed a taxi fare calculator using CSDI road network and information from the Transport Department.

https://www.td.gov.hk/tc/transport_in_hong_kong/public_transport/taxi/taxi_fare_of_hong_kong/index.html

Responsive version at: https://taxifare.hk

Thanks.

Live demo site: https://dashboard.hk/csdi-avgspeed.html

DATA.GOV.HK has some proof of concept dashboards using Government open data, here we will try to make the Traffic Speed Dashboard on our own server.

First, we will need to download the Road Network data from CSDI portal. Go to portal.csdi.gov.hk, and search for "Road Network", download the SHP format.

After unzip the file, the data inside is like the following, remember data in SHP file composes of a number of files.

For basemap we are using the CSDI Map API

https://portal.csdi.gov.hk/csdi-webpage/apidoc/TopographicMapAPI

There provide a sample source code base on OpenLayers v4, our development is based on that.

The Traffic Speed data is downloaded from Government Data One portal, the data is updated every 2 minutes (here we use the Processed Data).

We only need the CENTERLINE SHP file from the Road Network dataset, we will load it into the Postgresql database, which is an open source database. We installed that in an Ubuntu Linux server.

We use the ogr2ogr command to load it

ogr2ogr \
-nln centerline \
-nlt PROMOTE_TO_MULTI \
-lco GEOMETRY_NAME=geom \
-lco FID=gid \
-lco PRECISION=NO \
Pg:"dbname=gisdb host=localhost user=pguser port=5432" \
Transportation_TNM_20240605.gdb_CENTERLINE_converted.shp

For detail on how to install the Postgresql, please refer to the following post

https://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS3UbuntuPGSQLApt

We use a PHP program to download the Traffic Speed XML from Data One, combine with the geometry data from the Road Network, and generates a GEOJSON file which has both segments and the traffic speed information. The static GEOJSON will then be read by the OpenLayers UI for displaying the map data. An alternative for serving map data is to use GeoServer, but we try to make it simple in this exercise, we will cover GeoServer in future exercise.

<?php
// Load the XML file
$xml = simplexml_load_file('https://resource.data.one.gov.hk/td/traffic-detectors/irnAvgSpeed-all.xml');

// Check if the file was loaded successfully
if ($xml === false) {
    echo "Failed to load XML file.";
    foreach(libxml_get_errors() as $error) {
        echo "\n", $error->message;
    }
    exit;
}

// Connect to PostgreSQL database
$conn = new PDO('pgsql:host=192.168.0.108;dbname=gisdb','pguser','pwd');


// Build GeoJSON feature collection array
$geojson = array(
   'type'      => 'FeatureCollection',
   'features'  => array()
);


// Iterate through each book and print details
foreach ($xml->segments->segment  as $segment) {
	
	if ($segment->valid == "Y") {
	
		$sql = 'SELECT route_id, '.$segment->speed.' as speed, ST_AsGeoJSON(ST_Transform((geom),4326),6) AS geojson FROM centerline where route_id='.$segment->segment_id;
		
		// Try query or error
		$rs = $conn->query($sql);
		if (!$rs) {
			echo 'An SQL error occured.\n';
			exit;
		}
		
		
		// Loop through rows to build feature arrays
		while ($row = $rs->fetch(PDO::FETCH_ASSOC)) {
			$properties = $row;
			// Remove geojson and geometry fields from properties
			unset($properties['geojson']);
			unset($properties['geom']);
			$feature = array(
				 'type' => 'Feature',
				 'geometry' => json_decode($row['geojson'], true),
				 'properties' => $properties
			);
			// Add feature arrays to feature collection array
			array_push($geojson['features'], $feature);
		}	
	}
}

file_put_contents('/var/www/html/irnAvgSpeed.json', json_encode($geojson, JSON_NUMERIC_CHECK));

$conn = NULL;

?>

In Ubuntu server we set a cron task to run the above PHP program every 2 minutes

sudo crontab -e

***/2 * * * * /usr/bin/php -f /var/www/html/genjson.php**

Following is the HTML code we use for the map UI which is developed base on the above OpenLayers v4 sample code from CSDI. We created 3 map styles for the road segments to color, lime color for normal speed, yellow for medium speed, and red for slow speed. Following is the Javascript code which decides what style to apply to the segment features in OpenLayers.

        var styles = new ol.style.Style({
          stroke: new ol.style.Stroke({
            color: 'lime',
            width: 4
          })
        });


        var styles2 = new ol.style.Style({
          stroke: new ol.style.Stroke({
            color: 'yellow',
            width: 4
          })
        });

        var styles3 = new ol.style.Style({
          stroke: new ol.style.Stroke({
            color: 'red',
            width: 4
          })
        });
	  
		var styleFunction = function(feature) {
		  var speed = feature.get('speed');
		  if (speed > 30) {
			return [styles];
		  }
		  else if (speed > 17) {
			return [styles2];
		  }
		  else {
			return [styles3];
		  }
		};	

Following Javascript will retrieve the data from GEOJSON file (the static file generated by PHP above) every 30 seconds and update the map. Please note we need to convert the projection before overlaying it on map.

		var url = 'irnAvgSpeed.json';
		
		fetch(url, { cache: "no-store" }).then(function(response) {
		  return response.json();
		}).then(function(json) {

			var vectorSource = new ol.source.Vector({
			  features: (new ol.format.GeoJSON({
				dataProjection: 'EPSG:4326',
				featureProjection: 'EPSG:3857'
			  })).readFeatures(json)
			});

			var vectorLayer = new ol.layer.Vector({
			  source: vectorSource,
			  style: styleFunction
			});

			map.addLayer(vectorLayer);
			
			$('#datetime').text("Data timestamp: " + json.date + " " + json.time);
			

			
			setInterval(function(){ 
			 
				fetch(url, { cache: "no-store" }).then(function(response) {
				  return response.json();
				}).then(function(json) {
			 
					vectorSource.clear();

					vectorSource.addFeatures((new ol.format.GeoJSON({
					  dataProjection: 'EPSG:4326',
					  featureProjection: 'EPSG:3857'
					})).readFeatures(json));
					
					vectorSource.refresh();
					
					$('#datetime').text("Data timestamp: " + json.date + " " + json.time);				
				});
				
			},60000);
	
			
        });

Final result:

For full source code please visit our Github repository

https://github.com/geospatialhk

Any comments please email us at info@geospatial.hk

Thanks.

An event Application of Spatial Data for Entrepreneurs was held by Geospatial Lab and Cyberport Academy on 12 Dec 2023 at Cyberport.

Official link: https://csdigeolab.gov.hk/en/past-events/application-of-spatial-data-for-entrepreneurs

Latest trends and applications on CSDI (Common Spatial Data Infrastructure) was introduced by Government representatives and local startups.

Mr. Chan from Spatial Data Office talked about different kinds of data and their applications in CSDI. And the upcoming roadmaps like integration with BIM and real-time IoT data.

Mr. Chik from Lands Department introduced the latest dataset of 3D digital mapping projects, including 3D Visualization Map, 3D Indoor Map, and 3D pedestrian network, which open up new use cases. He also demonstrated Open3Dhk which is a free portal using the open data.

Official link: https://3d.map.gov.hk/

Mapxus is a local startup focusing on indoor mapping. Their Mr. Lau shared about different success cases of their applications, they have extended their business to other Asia countries such as Japan.

Mr. Wong from R2C2 talked about a platform for collaborating robots, with applications in construction and property management.

Polling was held during the event, such as the following.

I visited an exhibition about Common Spatial Data Infrastructure the other day at Festival Walk, showcases of 3D map and interactive flight sim games were prepared for the public to know more about spatial data. very interesting.

GeospatialTech Challenge 2021 final presentation was held by the Lands Department on 30 Jun. I was fortunate to be invited as a guest. Here is a highlight of the event.

The main theme of the competition is proposal for improvement of community facilities with the use of geospatial data and other open data, thereby promoting smart living and a healthy lifestyle. There was totally 4 student teams and 3 open teams.

Student Group

Team1: AIRUNNERS

Title: 健康稱心跑:AIRUN程式開發設計與應用設想

This proposal is base on the air quality project by HKUST called PRAISE HK. This app make use of the 3D Pedestrian Network from LandsD, public facilities data, traffic data, air quality data and land use data to plan for the route for street runners, based on their specific requirement.

Team2:Diocesan Boys' School

Title: Barrier-free Hong Kong

This study is based on the pain points about barrier-free access in Hong Kong, including limited barrier-free access, hilly terrain, insufficient geospatial data and sparse and often unavailable disabled parking spots. And propose solutions for that. This is a team formed by secondary school students but conducted a in depth analysis of the problem.

Team3: CUHK GRM

Title: 香港郊野公園火災危險警告 Hill Fire Alert System in Hong Kong Country Park

This app try to predict fire danger risk of different country parks base on real time data such as relative humidity together with slope, vegetation and aspect, and target to build a REST API for public use.

Team4: AED

Title: Addressing Emergency Demand Deriving a optimal placement and emergency management plan of Automatic External Defibrillators in HK

This team suggested a spatial analysis framework for matching the demand (based on demographic data), and supply of AED, and significantly improve the coverage of AED for saving human life.

Open Group

Team5: Varadise Limited

Title: Monitor Brownfield Sites and Unlawful Occupation of Government Land usage using 3D Photo-realistic model

This team developed a AI based geospatial analysis application using on convolution neural network, to identify unlawful occupation of Government Land usage.

Team6: Maphive Technology Limited

Title: Integration of indoor-outdoor navigation to improve city-walkability

This team developed an application to combine indoor and outdoor GIS, using WIFI fingerprinting and sensor fusion for indoor positioning, and provide app for indoor outdoor navigation.

Team7: Smart Life, Smart Live

Title: Enhancement of smart mobility and public health through mobile application

This team developed app to track health of the users, and control calorie intake and consumption base on tracking the diet and exercise of the users.

For more information please visit the official site

Lands Department - GeoSpatialTech Challenge

GeoSpatialTech Challenge

GeoSpatialTech Challenge