Cross-Domain Ajax Request

Hi everyone. I’m Tania Khalafbeigi, a new PhD student in this lab and this is my first blog post. In this post I want to share a problem-solution with you that I faced during implementing two projects front-ends. The problem is with cross-domain Ajax request. The explanation of this problem and its solution will be discussed in the following paragraphs.

Cross-domain Ajax requests are requests that are sent from one domain to another domain. For example, if we send an Ajax request from our domain http://abc.com to another domain http://xyz.com, this would be a cross-domain Ajax request. This kind of request is restricted in the browser because of the same-origin policy of the browsers. The same-origin policy in browsers dictates that certain types of data transfer in the browser layer (via JavaScript) must be restricted to only occur if the target resource’s domain is identical to the page making the request. This policy is in place in all modern browsers to help protect users from unwanted or unsafe malicious JavaScript behaviors. There are different solutions for this problem which I want to discuss three of them in this post.

The First solution is for the case that we have access to the remote server that we want to access through cross-domain Ajax request. In this case, we can change the response header of the server and add “Access-Control-Allow-Origin:  http://abc.com to it. With this change a client from http://abc.com can send a request to that remote server with no problem. In other words, by adding that feature to the response header, the server tells the browser that the domain http://abc.com has the authority to send the request to it. Moreover, if we set “Access-Control-Allow-Origin:  * in the server response, we allow all other domains to send us Ajax requests. In my opinion, this solution is the simplest one.

The second solution is using JSONP which stands for JSON-with-Padding. We need to have access to the server for applying this solution as well. Also, this solution has the limitation that the response type could only be JSON. The basic idea of JSONP is that the JSON response is wrapped in a JavaScript function so that it can be accessed through a <script> tag. In this case, instead of sending an Ajax request to the server, we add a <script> tag to our html file and set its source to the server URL. As an example, suppose that we want to get this JSON response from the server:

{
   ‘id’: 1 ,
   ‘Name’: ’foo’ 
}

For using JSONP, instead of sending this response to the user, we wrap it in a JavaScript function call and send it to the user  like this:

callback({‘id’ : 1 ,‘Name’: ’foo’});

As a result, when a client wants to send a request to this server, it only needs to add a <script> tag to its html source like this:

<script type=’text/javascript’
        src=’http://xyz.com?jsonp=callback’>
</script>

There is one more thing we need to do and that is defining the callback function in the JavaScript code of the client. This can be a code example of the client:

<script>
   function callback(json){
     alert(json.Name);
   }
</script>

As you see, this solution needs changing both the client and server of the request.

The last solution that we discuss in this post is for the case that we don’t have access to the server. In this case we can access the server response using a proxy. This is a sample code of the proxy in php language:

// Get that server's content
$handle = fopen($serverURL, "r");
// If there is something, read and return
if ($handle) {
   while (!feof($handle)) {
      $buffer = fgets($handle, 4096);
      echo $buffer;
   }
   fclose($handle);
}

So, these are three useful solutions for sending cross-domain Ajax requests. In three month of working I faced this problem twice. I hope this blog can help speed up handling this problem.

References:
[1]    http://en.wikipedia.org/wiki/Cross-Origin_Resource_Sharing
[2]    http://en.wikipedia.org/wiki/Same_origin_policy
[3]    http://en.wikipedia.org/wiki/JSONP
[4]    http://json-p.org/
[5]    http://msdn.microsoft.com/en-us/library/dd573303(v=vs.85).aspx

Introduction of our web mapping system for arctic region.

Hey, guys! My name is Han-Fang (Fanny) Tsai, a master student in GeoSensorWeb lab since January 2012. My research focuses on development of web mapping system, in particular for Polar Regions. I have participated in a GEOIDE project: Improved Global Web Map Visualization Project (Our project website: http://www.yorku.ca/ssii109/project.html).Since this project ended in couple of months ago, as I was involved in this project as a system developer, and we successfully delivered a preliminary prototype, I would like to talk about what I have done for this project.

This project is a cooperative work between three universities (University of Calgary, York University, and Laval University) and other government and industry partners (e.g. natural resource Canada, ESRI, etc.). The goal of this project is to develop an improved worldwide web mapping system to store and present pre-rendered geographic information. So the questions are: why do we need a better web mapping system? And what are the drawbacks of current web mapping system?

The biggest problem is the map projection that most of current web mapping sites are using: web Mercator. Since Google published Google Maps in 2005, web mapping have been widely applied in many research fields. Also all major web mapping applications/services are using web Mercator projection for their base map ever since. However, many articles have argued that maybe web Mercator is a poor choice?

As you may already know, web Mercator has following characteristics:

  • There is a slightly difference between web Mercator and Mercator projection: web Mercator is not a conformal projection [ref], thus it cannot preserve either shape or direction, and it only fits for visualization on the Internet.
  •  Web Mercator cannot show the entire world, the maximum latitude shown is approximately 85.05 degrees.
  • In web Mercator projection, the distortion increases when latitude increases and goes to infinity at the poles.

Thus, web Mercator is a bad choice for high-latitude region since it crazily distorts the area in high-latitude region. As the Arctic researches become increasingly important as places for abundant natural resources and environmental studies (e.g., global warming and climate change), Arctic researches attract more and more attentions. Therefore, a Mercator-projection-based web mapping system apparently cannot be used for Arctic researches; as a result, a better web mapping system which is suitable for polar region is necessary.

We have developed a web mapping system with multi-polar-projections. We have a server-side component that pre-renders base maps in different projections and parses the vector data users upload. Then, we present data in multiple polar projections in the client-side online platform. In addition, we found that polar projections are difficult to interpret when viewing local areas because the North does not face up like that in the web Mercator projection. Therefore, we propose a new module allowing users to rotate map and visualize geospatial data sets in the polar region. Hence, wherever area users zoom into, users can always rotate the map to let the North facing up. The prototype demonstrates an efficient and intuitive web mapping system for the Arctic region.

Untitled

You can give it a try at: http://tsai.geocens.ca:8080/TileServer/

A Simple Attempt to Display Sensor Data with D3

Hi everyone, my name is xiaohong shang, a new Msc students in Geosensor Lab. This is my first semester in University of Calgary and the first blog here. Because I just came here in January, I have little to share with everyone about my research now. But I am taking two classes right now, so I am going to talk something about one of my class and the class project.

One of my classes is ENGO 551 named “Advanced topics in Geographic Information System”.  This class is the 4th year undergraduate level. But I think it is also a good option for the new graduate student. There are two parts in the class: the lecture part which cover the advanced topics in GIS area and the labs part which is very straight forward to teach students how to program with Javascript to develop map application. There will be a final project in this class.In this blog post,I will talk about some initial ideas regarding my final project.

My project topic is “Display sensor data with D3”.To load sensor web data I will use the GeoCENS platform. GeoCENS is an application to search and display environmental sensing data like water, wind etc. For more detailed information about GeoCENS, you can reference the previous blog posted byAlec(http://sensorweb.geomatics.ucalgary.ca/blog/?p=194).D3 stands for Data Driven Document, which is a javascript library for manipulating documents based on data. From D3 official website: “D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation”. There are lots of data-driven examples developed by D3. In  my course project, I would like to explore how can we apply D3 to display various types of sensor data.

I am working on this class project right now. Hopefully I will post more details and the final result later.

GeoCENS website: http://dev.geocens.ca/bingmap_browser

SensorWeb PivotViewer

Hello everyone! Please forgive us for our lack of maintenance of this blog. We are going to post more starting from this one.

As this is my first blog post, I am going to give a brief introduction about myself. I am Alec Huang, a PhD Candidate in this lab. I joined this lab in July 2009, and have participated in many projects, such as Geospatial Cyberinfrastructure for Environmental Sensing (GeoCENS), Water and Environmental Hub (WeHUB), Open Sensor Data Map, OGC Sensor Web interface for Internet of Things (SW-IoT) Standard Working Group, and many other small projects. If you are interested to find out more about my work, you can visit my profile page on our lab website (http://sensorweb.geomatics.ucalgary.ca/alec-huang).

In this post, I am going to focus on one of my side projects: the SensorWeb PivotViewer.

World-Wide Sensor Web and World-Wide Web (WWW) are very similar in many ways. (Especially when we discuss in the context of OGC SWE) Think about when we try to find information on WWW every day. The first thing we do is open our web browser (e.g., IE, Chrome, Firefox, etc.). Then we go to search engine (e.g., Google, Bing, Yahoo!, etc.) if we are looking for something on purpose. Otherwise, we may go to some web portals (e.g., Yahoo!, MSN, CNN.com, reddit.com, etc.) if we just want to “browse the web”. Finally, either through search engines or web portals, we can simply click on the links of web pages and start viewing the contents. It’s simple and intuitive.

For the World-Wide Sensor Web, we envision the same simplicity on discovering and consuming the sensor web contents. However, there are some missing pieces to realize this vision. For example, we need a “Sensor Web Browser” to visualize the content on the sensor web. We also need a “search engine” and/or a “web portal” for users to discover the sensor data they are interested in. So this SensorWeb PivotViewer project (and other projects in our lab, such as GeoCENS) is trying to create a cyber-infrastructure for users to discover and consume the World-Wide Sensor Web content in an easy and intuitive way.

sensorweb pivotviewer

In this project, we apply two Microsoft tools to facilitate the sensor data discovery and visualization. The first tool we use is the Silverlight-based PivotViewer (http://www.microsoft.com/silverlight/pivotviewer/). PivotViewer is a great data discovery tool for users to interact with massive amount of data. With its multi-facet-filtering and Deep-Zoom functionalities, users can efficiently find the data they are interested in. In this project, we treat each sensor data instance in OGC Web Services (OWSs) as a data item in PivotViewer. (Note: currently, we only support the SOS observed property layers and the WMS layers). By converting OWS metadata into PivotViewer facets, users can discover the sensor data instances through keyword search, facet-filtering search, or data item browsing. PivotViewer has other features that facilitate the discovery process, such as the image tiles (provide a preview of dataset) and column view. You can visit the PivotViewer official website to learn more about its functionalities.

The second tool we use is the BingMaps (http://www.bing.com/maps/). BingMaps, which is very similar to the GoogleMaps, allows users to intuitively browse geographic data. While sensor data are geographic in nature, BingMaps is a good platform for visualizing sensor data. By giving BingMaps the ability to retrieve and display data instances/layers from OWSs, it becomes, what we called, the Sensor Web Browser.

As the PivotViewer handles the data discovery and the BingMaps handles the data visualization, we also integrate these two modules together. So users can first use the PivotViewer to find the OWS data instance they are interested in, and click on the “Add Layer” button on the top-right corner of the image tile to add the data layer onto the BingMaps.

I also recorded a video for this system, which was listed as one of the finalists of the AITF Summit Student Video competition. In this 3-mins video, it briefly explains the motivation and demonstration of this project. In the demonstration, you can see how to use this SensorWeb PivotViewer.

To sum up, we believe that the data discovery and data visualization/consumption are big challenges of the World-Wide Sensor Web. The SensorWeb PivotViewer demonstrates one possible solution that we can easily consume the sensor web content like we consume the WWW content every day. Of course, I didn’t mention all the technical details in this blog post, such as where we get the OWS data instance list and how we improve the performance of retrieving sensor data, etc. Maybe we will talk about those in the future post. Stay tuned!

Project website: http://sensorweb.geomatics.ucalgary.ca/sensorwebpivotviewer/

Prototype: http://dev.geocens.ca/pivot_viewer

TrafficPulse: a new solution for transportation

I believe that most of us suffer from traffic jams. Traffic congestion is one of the main problems that modern day cities face, and traffic congestion in the United States are costing Americans $68 billion each year in wasted time and fuel, according to a new report. Whenever I was stuck in traffic, I wondered if there are better ways to solve the problem? I knew that I could always turn on the radio and wait for the traffic report every 15 minutes, however, for me it’s annoying that I need to wait and filter out the information not relevant to me.

So there was a thought, it would be nice if I can query a system for traffic details, as well as visualize current traffic from my vicinity with a few clicks on my Smartphone, so that I can escape from the traffic. Luckily, I was working under TrafficPulse project and was attempting to create innovative solutions for this.

We developed TrafficPulse, a complete front-end and back-end system that is designed to solve traffic problem. The front-end is a mobile application (TrafficPulse APP) and back-end is a cloud-based data storage that uses spatial indices to increase query performance. You can find details of our cloud storage in a journal we published. (http://sensorweb.geomatics.ucalgary.ca/gsw/geopot-cloud-based-geolocation-data-service-mobile-applications). TrafficPulse APP is an easy-to-use mobile application, which allows users to publish traffic information, as well as to query and visualize traffic dynamics. Users can provide traffic information at different level of details. For example, they can simply click buttons to report different traffic conditions, or they can provide text contents (which provide more details). Users can as well share traffic details through photos. Not only can users publish traffic information, but they can also query/visualize near-by traffic and latest traffic events around. To bring some “coolness”, I even developed an Augmented Reality feature that navigates users to bus stops.

You may wonder where is the data going? All the information shared are stored and indexed in the back-end server. Since we aim to benefit users by providing services derived from user-shared contents, server responds users’ requests by analyzing data collected. For example, we developed a carpooling recommendation system based on the data collected by TrafficPulse (http://sensorweb.geomatics.ucalgary.ca/gsw/crowd-sourced-carpool-recommendation-based-simple-and-efficient-trajectory-grouping).

PUCK Over Bluetooth

Recently, in the area of sensor networks, the topic of interoperability has become the focus of many researchers’ attention. The Open Geospatial Consortium (OGC) is an international industry consortium that has been supporting geospatial interoperability since 1994, and has recently published a new standard, PUCK. PUCK is a standard defining a protocol for RS232 and Ethernet connected instruments. Simply put, PUCK enables outside devices to access a sensor, including the sensor’s datasheet, metadata, driver code and so forth.

Bluetooth is a wireless technology standard which has empowered devices to transfer data over short distances. Today, we see Bluetooth technology everywhere in: cell phones, computers, notebooks, and now sensors. By attaching a Bluetooth modem to a sensor, we can dramatically change our notion of sensor networks. Since the Bluetooth modem transfers data to the serial port of the device its connected to, we can integrate PUCK protocol, which was originally defined for communication over cables, into Bluetooth. By combining Bluetooth and PUCK, we have a standardized and automated way for sensors to communicate wirelessly and in real time. In summary, combining Bluetooth and PUCK has the following benefits:

  • Sensor plug-and-play (very little or even no configuration needed)
  • Discoverable (sensors can be discovered, paired and consumed wirelessly)
  • Flexible (the Bluetooth wireless capability allows users to install the sensors anywhere)
  • Interoperable (based on open and international standards)

To implement PUCK over Bluetooth, we chose Netduino Plus as a sensor compatible
SBC (Single Board Computer) and BlueSMiRF Silver as a Bluetooth modem for the Netduino Plus.

The current PUCK standard doesn’t define any commands for the sensor reading, so we developed a new protocol to find out what the value is being measured by the sensor. Based on this new protocol, the user should first send a “GETCAPABILITIES” command to gather information about sensors IDs, phenomenon IDs, and unit of measurements. Next, the user sends a “GETREADING” command followed by the sensor ID and number of readings which has been recorded on the device. These two commands and sensor description are defined in a SensorML file which is a Sensor Interface Descriptor (SID) XML file recorded on the device memory.

Some software can be integrated into our developed infrastructure that helps the sensor data to be published on the Internet. For example, PUCK detector (see the Figure) is a software that automatically sets the appropriate configurations (serial port number, baud rate, parity, etc.) to establish the Bluetooth connection. We also need another software to download the SID file. To do this, we use SID Interpreter, which is applied to retrieve the SID file, extract its information, get the sensor measurements, and finally send the observations to the Internet (via Sensor Observation Service).

Bluetooth enabled devices (cell phones, computers) are a part of our daily life. By implementing PUCK over Bluetooth, we can easily make sensors a part of tomorrow’s daily life.

Interoperable And Tiny Web Service For Sensors

The “Internet of things” (IoT) is currently one of the most exciting topics in information and communication technologies (ICT). IoT is the next generation of the Internet where each object in the physical world can have a virtual representation in the digital world, these objects can communicate with each other via computer networks.

http://www.gizmag.com

Nowadays, IoT networks are dramatically spreading in size and number. Ericsson, a leading manufacturer of electronics, has predicted that there would be 50 to 60 billion Internet-connected devices by 2020. To really take advantage of the IoT, all these devices (sensors or actuators) need to be able to communicate with one another. Since these devices and the way they want to talk are very different from each other, the network of IoT would be too heterogeneous. Therefore, the interoperability between the devices should be focused more than other aspects.

Regarding the interoperable viewpoint, we would like to build a standard-based platform that each node (e.g. sensor) on the network can easily “discover”, “access”, and “interact with” the objects around itself. As a result, the sensors should be able to send messages to or get requests from other devices. To make this communication which follows a Client-Server network topology, embedded computers (to attach sensors) and HTTP protocols can play the main role.

To achieve this goal, I developed a “tiny web server” for an embedded computer that supports sensors. This server has less functionality, responsibility and code space compared to regular web servers. For sensor discovery on the network, I considered two possible architectures:

  1. Using a centralized gateway: In this architecture, all sensors are automatically connected to a gateway and others can indirectly communicate with them. Then, registered sensors to the gateway are able to update their records on a repository frequently. Moreover, users can go to the gateway to see the remote sensing around themselves, or to send requests (e.g. change frequency of reading) to the sensors/actuators.
  2. Using a direct connection: An alternative architecture to decentralize the sensors’ integration is earned by setting a static unique IP address for sensors. In this case, the network nodes can directly communicate with the sensors with no need to visit a gateway.

The most noticeable part of my project is the use of standard protocols. These protocols have been defined by OGC (Open Geospatial Consortium) for sensors with large resources, not for small sensors with embedded computers. Furthermore, these standards not only empower the interoperability aspect, but also make my project different from the current works done in the area of IoT. I am currently writing a research paper on this topic, and hopefully will be accepted for publication soon. Stay tuned!

Virtual Layers by Automatic Text Matching

My research has been focused on data mining the past few months. How does this relate to data interoperability? Let me explain.

Ultimately we want to design a system that can connect to as many data providers as possible. The goal of this is to allow users to browse available data sources, so they can find what they need. Our users would be scientists, researchers, or just very keen individuals.

Specifically, I work with data collected from a SOS, which is a type of service that provides sensor data. We are interested in building systems that can connect to tens, hundreds, and maybe in the future millions of SOSs. However, problems arise with something as simple as wanting to select “wind speed” data.

The data is structured in a way that there are no specific naming conventions. Some people use the term “Wind Speed”, “Speed”, “Wind Direction”, “winddirection”… the syntactic variations of the terms make it impossible to perform basic string matching.

We can implement approximate measures of string matching, but this ignores semantic relationships between similar concepts, such as “rain” and “precipitation”. Not only this, but we find that the label of the data (or the title) has other irregularities, such as being a URI or maybe even represented in a different language.

So I’ve been investigating data mining techniques to aggregate similar data layers so we can design systems for easier data exploration. Our first system approach has been to normalize and tokenize the data layer names into keywords. We then apply a matching algorithm from a unique observed property dictionary to these keywords. By using our dictionary as a virtual layer list, we can automatically retrieve all similar data layers for the user.

A Virtual Layer List

The current work in progress can be found here: http://ben.geocens.ca/virtualsos

I will continue to look at various algorithms for clustering and classification to group similar data layers. I feel that these techniques can automate the construction of a data catalogue,  and also help solve problems of missing or incomplete metadata.