Week 3 – Context and APIs
Mobile and Ubiquitous Computing 2020/21
Sandy Gould, School of Computer Science University of Birmingham
Overview
This week we will look at two topics in mobile and ubiquitous computing. First, we’re going to focus on context aware systems. Then we’re going to look at web APIs. We will cover:
− The necessity of context awareness as we lose control in ubicomp environments
− The difficulties for machines of creating contextual awareness
− Types of context that machines must be able to sense to be context aware
− Features of context aware systems
− Why we use web-based APIs
− Service design patterns (focus on REST and JSON)
Important concepts
We have a split focus this week, but there is still a lot of very important content here, with critical concepts that we will be coming back to throughout the module.
Embeddedness, control and context awareness
Why is context awareness a big challenge in mobile and ubiquitous computing? Think back to Week 1 where we discussed the long-term evolution of computing from mainframe to desktop to ubiquitous computing. As devices become embedded into our everyday lives, designers have less control over the kind of contexts that they will be used in. Think about that mainframe era – the developers in this context have complete control. They know what kind of technical environment the machine will run it; the network connections it will have, the other systems it will be plugged in to. They knew what kind of physical environment it would be housed in; the temperature, the size of the room, who would have access to the machine and how it would be handled.
In the ubiquitous computing era, these things no longer hold true. Devices must be capable of fitting into whatever technical (heterogenous execution environment) or physical environment (heterogenous environment) the device finds itself in. To be able to behave appropriately, devices need to be able to have context awareness; they need to understand the world around them so they can adapt to it. This is a really big challenge for ubicomp – as people we take for granted how our awareness of context which actions we perform and which we avoid in a given scenario. Because we take this skill –to instantaneously recognise a situation and select appropriate behaviours– for granted, it is often difficult for us to articulate why one thing is appropriate in a given context and another is not. We don’t always get it right either, which is often how we end up embarrassing ourselves. If we find it difficult to describe how we deal with context and we make mistakes too, is it any wonder that context awareness in machines is still in a very early state?
Features of context aware systems
Dey (2010) explains the features a system needs to help create context awareness. They settled on three concepts: presentation, execution and tagging.
Presentation is the idea that context aware systems should be able to show information that is relevant to people in their current context. For example, your phone could prioritise certain kinds of notifications if it realised you were on a night out to when it knew you were in the library trying to study.
Execution is the idea that context aware systems should be able to perform tasks on your behalf when a given context is detected. For instance, your phone could automatically reply to messages you received whilst driving.
Tagging is the idea that events observed by a system need to be ‘tagged’ with context so that the context information can be processed and used to help determine context in future scenarios. So your
phone could keep track of how quickly you respond to all of the notifications you receive and the content of those notifications. This information could be combined with other contextual information to ‘tag’ the notifications. This tagged data could be used by rule based or probabilistic systems (e.g., machine learning) to help infer context in future.
These three features help us to think about the critical components for a context aware ubicomp system. As we will see later, it is not necessary for a given system to have all of these aspects – but it will have at least one of these aspects.
Applied context awareness in real scenarios
People have tried to implement context aware systems in a number of real scenarios. First, let’s consider the medical context. This is an environment where small errors can be very significant. Bardram and Nørskov (2008) developed a context aware system by augmenting an operating theatre environment with tools that had RFID embedded in them. The goal of this system was to avoid ‘Three W’ errors: wrong place, wrong procedure, wrong patient. The idea was that the system could track all the components of the system and, with a rule-based understanding of which things should be collocated (correct patient in the correct theatre) and which shouldn’t (say, the wrong surgeons for a given patient) such a context-aware system could help healthcare professionals realise when something has gone amiss. The system was deployed in practice, but that does not necessarily mean it is practicable: there was a lot of technology to deploy and the context awareness requires a complex system that would need adapting for each context (due to heterogenous execution and physical environments).
Context aware systems have also been deployed in agricultural contexts, too. Agriculture is already highly computerised, so much so that farmers are employing Russian hackers to break DRM on their tractors. Getting all of the existing technology to work together is a big challenge though. There is a huge variety of equipment and the physical context is challenging. Kjær (2008) discusses developing a middleware that would allow independent agricultural technology to communicate. They point out the challenge of poor network coverage in agricultural areas, the non-standardisation of equipment (see Week 1) and the limitations of GPS (see Week 4) as being major barriers to integration.
Designing for context
We have learned so far about why we need context aware systems, the features context aware systems have and a couple of examples of context aware deployments. How can we design systems with context awareness? Anind Dey’s chapter (Dey, 2010) in John Krumm’s Ubiquitous Computing Fundamentals (Krumm, 2010) proposes five steps for building a context aware system:
• Specification, in this step a designer works out what it is that they want to do. For instance ‘show museum visitors information tailored to their journey through the museum’. In this step we are simply articulating what the system should do when it is complete.
• Acquisition, in this step a designer needs to specify how they will collect the information that they will use to provide observations to their system. So in this stage it might be ‘develop an RFID-based token system to detect which exhibits a visitors is at’.
• Delivery, in this step a designer must work out how the acquired information will be distributed and shared to relevant systems. This could use a publish/subscribe model. For example, ‘use MQTT to deliver information on RFID inputs to subscribing systems’.
• Reception, in this step a system architect must decide what the threshold for notification is. For something like an RFID-based solution you might want to receive all notifications of changes in state. For, say, machine vision applications, it might make sense to have a threshold (e.g., size of movement) so only relevant observations are funnelled through the system.
• Action, this is the final step where the system analyses the input and decides how to act (e.g., display information about dinosaurs if someone has just come from the bird exhibit).
Salber et al.’s (1999) context widgets approach also describes some principles for the engineering of context aware systems which, they say, allows for better, more robust systems to be developed.
They talk about the necessity of hiding the complexity of sensors from developers. This is critical, and has been one of the great successes of APIs on Android and iOS. If, for example, you want to know when
there has been a sudden movement in an Android application (say for a dash cam application), rather than have to continuously receive accelerometer data and determine, based on noisy sensor input, whether there has been a significant movement, the API allows you to subscribe to such events. All of the complexity is abstracted away. We will discuss this in more detail in Week 4 when we talk about GPS.
Salber et al. also describe the importance of information being abstracted in appropriate ways. In a system designed to let people know which room their colleagues are in, would it make sense to give longitude and latitude information? No, it wouldn’t the useful unit of analysis here is the room, so raw data should be abstracted into this form to make it more useful for developers. This is critical when designing APIs to access a service – what is the useful unit of information? In most cases it is not raw sensor data.
Finally, Salber et al. recommend the reuse of existing libraries and techniques. Context can be idiosyncratic, but a lot of the fundamental challenges of sensing and interpreting data are similar across domains. This is another reason for the success of, for instance, AR/VR techniques in mobile devices in recent years. Shared, common libraries (that are well documented) allow people to quickly make use of tools and focus on the application domain.
In all this though, we must think back to the concept of seamfulness (Chalmers & MacColl, 2003) that was introduced in Week 1. There will always be a situation in which these kinds of systems fail. Something will go wrong, whether because you as an engineer have failed to anticipate some mode of use or because another engineer (writing the libraries you use, for instance) has failed to do so. This is why the concept of seamfulness is so important – how will the system you’re designing behave when things do go wrong? Refer back to Week 1 for the definitions of the different approaches that Chalmers and MacColl suggested.
Introducing web-based APIs
As we have already learnt, the ability for devices and services to communicate with one another is a critical component of ubiquitous computing systems. Last week we saw how ultrasound could be used to transport information between devices. But most things don’t communicate like this. Web-based APIs are probably the most popular way to transport information. An API is a way for applications and services to communicate in a standardised way. They are how we get our apps to talk to the underlying Android operating system (not all of the API is made public and documented; there are proprietary aspects to APIs not intended for use by apps). Web-based APIs make use of existing technologies like HTTP to facilitate the movement of information. This makes it much easier to ‘hook into’ existing APIs or to develop your own.
There are many design patterns and protocols for sharing information between applications and services. SOAP is a messaging protocol that uses XML to transmit data. MQTT is used extensively in low- bandwidth, low-power contexts. These protocols provide the means of transmitting information. But there is also a bigger picture, which is focused on the interoperability of services (just because two services use HTTP doesn’t mean they can talk to each other). REST, a service design pattern, is a way of structuring communications using HTTP in a way that allows developers to very quickly learn and assimilate new APIs into their software development practice.
REST
REST, or REpresentational State Transfer is our focus because it is in common use. It is does not solve all problems. In many instances it creates a huge number. But it is pretty much the standard way of designing APIs that use HTTP. There are lots of other approaches but a complete circumnavigation of service design is well beyond the scope of this module.
REST is a stateless approach to service design. This means that it is ideally suited to the stateless nature of the web infrastructure that it often runs over (e.g., proxies). Stateless means that each request is treated as a new, independent request (there is no intrinsic carry-over between requests). The REST approach scales well, because challenges of, say, load-balancing are well understood in this context from the development of high-performance webservers.
REST revolves around URIs, the labels used in them and the actions by which they can be accessed. It makes use of HTTP verbs. These verbs imply different kinds of actions and these semantics are important to signal to developers using an API what to expect. The HTTP verbs most commonly used are.
GET POST PUT
Retrieve some information for me Create or update some information for me
Create or update some information for me
Other verbs are available, but not widely used: PATCH, DELETE, OPTIONS, HEAD, TRACE, CONNECT.
These verbs allow RESTful services to follow the CRUD principles of being able to create, read, update and delete data.
The GET verb is used to retrieve (read) information from the system. They are encoded in the query string of an HTTP request and look something like this:
http://myapi.com/people/getPeople?sort=desc&limit=10
You will be familiar with these because most of the requests you make from your web browser are of the GET variety. GET requests should not be used to change the state of the system at all. They should only retrieve the current state of the system (or the data the system controls).
The POST verb is used to send information to effect some action in the system. Normally a request would look like this:
http://myapi.com/mailing/addNewRecipient
And the payload, the information being sent, is sent in the body of the HTTP request, unlike GET requests where the request is made in the query string (and hence in the head of the request). Note that in this instance, the action, addNewRecipient will be expecting new information in the system, not updating old information. You will be most familiar with POST from forms in your browser (e.g., Google Forms) which use POST to return information you have entered to a server.
Finally, there’s the PUT verb (as noted there are other verbs but they are very rarely seen). These, like POST requests are in the body of the request and they look something like this:
http://myapi.com/templates/unique-template-1024
Note that with PUT, unlike POST, we are not specifying some action in the system to which we wish to push our data toward. Instead we are specifying a unique item in the API. Why? The idea of PUT is that it should be used for creating or directly updating explicitly defined resources (by URI). In this case, providing the data that for resource unique-template-1024 either as a new resource or to update it existing one.
The use of these verbs is controversial if you look at a lot of popular APIs you will see that they do not use PUT requests. Some people make no distinction between PUT and POST and say that you should just use POST for sending and GET for receiving. It’s also the case that for big APIs, like Twitter, the API is never going to expose direct access to resources that allow data to be replaced. Instead new items (e.g., update to post #12903729) are created, rather than data being overwritten. Because we are dealing with a philosophy for design and not a set of technical constraints there is naturally disagreement over what best practice looks like. Keep this in mind as you look at public APIs. One good guide is that if your system manages all aspects of creating and updating (using actions like createNewRecord or updateOldRecord) then POST might be more appropriate. If your system allows individual resources to be explicitly created or updated then PUT might make more sense. But there is no golden rule.
An important characteristic of REST APIs is idempotency. This is the idea that an action should have the same effect no matter how many times it is run. This is important over unreliable connections and systems like the internet where the order of delivery is not guaranteed or requests may have to be resent. If we have designed our API to be idempotent, it makes our life much easier as developers
because we don’t need to worry about multiple requests – the state of the system should be unaffected. Compare this to the completely non-idempotent experience of webforms on online shops which tell you that if you click a button more than once you might be charged more than once.
REST is not a magic solution to inter-service communication. There are many pitfalls to using it. First, the data returned by a REST API should be the same every time, as specified by the documentation. This means you often have to pull-through far more information than you actually need because there is no specific endpoint for the information your require. So either you end up with too much information, your API accepts increasing numbers of parameters to an endpoint (scope-creep) or you end up with an impossible number of endpoints. Alternatives to REST, like GraphQL offer much more fine-grained control over the data that is requested.
Webhooks
HTTP is a stateless protocol. We make a request, the server responds and… that’s it. It all vanishes, transaction over. There is no memory for what came before and nothing to imply what will happen next. The design of HTTP means that there is nothing in the protocol that can support something like a publish-subscribe module used by protocols like MQTT which allows consumers of a service to have updates pushed. On the web this historically has meant techniques like ‘long polling’ have had to be used to receive timely updates. When we’re talking with an API though, we don’t even have the option to use long polling because once the connection is open it’s possible that there will never be another event. No good at all.
So how do we get updates from APIs? Do we just have to poll the API at regular intervals to find out if anything has changed? That’s not very efficient at all. If there are frequent updates then out requesting service will be constantly outdated. If there are very irregular updates then we’re wasting resources polling to get the same result time after time.
The solution is to use Webhooks. These allows us to simulate a publish-subscribe approach with a web API. We tell an API that supports Webhooks what kinds of events we want to get. Then the service, when updates occur, sends a POST request to an endpoint that we specify. This means that our service just needs to be ready and waiting to receive these POST requests, which it can process (and potentially use as a prompt to fire off more API requests). This solves the polling problem, but it does require more complexity for the API maintainer to keep track of which endpoints need to be notified of particular events. It also means that we need to have an endpoint that is exposed to and is accessible from the internet.
JSON
Although REST is very much tied to HTTP, the payload of a service (i.e., what it returns) is not specified. A REST service could return XML. It could return some other plain text. It could return binary blobs (and they often do, like images or videos). Often we want to return some data from a service. A list of messages. The messages sent by a given user. Whatever. A now standard way of sharing structured data of this kind is JSON. JSON started as a subset of Javascript. It is flexible, has a relatively low overhead and the main reason for its popularity is that is very readily interpreted by browsers (by virtue of being a subset of Javascript). It is now used extensively outside browser contexts as a way of packing data. It’s easy enough to read.
JSON is not perfect either, though. The flexibility it has mean it lacks rigour. You don’t have types and although simple documents are easy to read once you have nested arrays and objects to the n-th degree it’s hard to follow what’s going on. There are approaches to solving the rigour issue (e.g., JSON Schema), but they are not intrinsic and so are not always used.
Good APIs
If you ever come to use APIs in a professional context the first thing you will notice is that they vary very substantially in quality. Good APIs should have descriptive endpoints so that someone using it should have a reasonable idea what they will get back from a given request. They should also be stable. APIs exist for other code to build on. Any changes should be made very infrequently and only with a very long notice period (years). Otherwise you risk destabilising applications you don’t even realise are using your
API (dynamic heterogenous execution environment!) Finally, APIs should be consistent. If you use PUT in a certain way for an endpoint, all your other endpoints should use it in a similar way. If variables across the API refer to the same objects then you should use the same variable names across the API. It sounds simple but often people do not do this, especially in smaller/bespoke APIs. There are often long lists of what makes a good API (Bloch, 2006).
API-First design
API-First design is an approach to software development that makes the API the central feature of development around which other aspects of development are oriented. There are some advantages to this approach. First, starting out from an API tremendously increases the ease of building software down the line. You might only want to have a website for now, but if you have a high-quality well documented API then building a phone app is easy. Maybe down the line, other apps will be able to consume aspects of the API. If you already have an API in place, opening up access becomes a lot easier. Focusing on producing an API also means that, as a developer, you have to focus very early in the process on what your system is going to represent; it forces you to be careful early on about the names you use for things and what a given name comprises. It’s a little like designing a relational database in that it forces you to consider the fundamental aspects of what your software will do. This is helpful for ironing-out conceptual problems that might be seriously problematic if discovered further along in the process.
Of course, this approach isn’t magic. APIs, as we’ve seen, should be stable. This means that, in an API first approach, once you have committed to an API it gets progressively more and more difficult to make changes. So once an API has been made final it really really really better do what it needs to. There’s also an idea that the API-first design allows a ‘clean’ approach to exposing functionality. Existing services should be adapted and extended to produce whatever the API requires. This is nice in principle, but it has the potential to force square pegs into round holes and significantly increase the complexity of what happens ‘under the surface’.
An API-First approach to development is particularly useful in the context of mobile and ubiquitous computing, though. In this context, the way that a system is going to be used, appropriated, in future is hard to know. Being API-first gives maximum potential for a variety of devices to interact with a service.
References
Bardram, J. E., & Nørskov, N. (2008). A Context-aware Patient Safety System for the Operating Room. Proceedings of the 10th International Conference on Ubiquitous Computing, 272–281. https://doi.org/10.1145/1409635.1409672
Bloch, J. (2006). How to design a good API and why it matters. Companion to the 21st ACM SIGPLAN Symposium on Object-Oriented Programming Systems, Languages, and Applications, 506–507. https://doi.org/10.1145/1176617.1176622
Chalmers, M., & MacColl, I. (2003). Seamful and seamless design in ubiquitous computing. Workshop at the Crossroads: The Interaction of HCI and Systems Issues in UbiComp, 17. https://www.researchgate.net/profile/Matthew_Chalmers/publication/228551086_Seamful_and _seamless_design_in_ubiquitous_computing/links/0c9605188da3cf3173000000.pdf
Dey, A. (2010). Context-Aware Computing. In J. Krumm (Ed.), Ubiquitous Computing Fundamentals (pp. 321–348). CRC Press.
Kjær, K. E. (2008). Designing Middleware for Context Awareness in Agriculture. Proceedings of the 5th Middleware Doctoral Symposium, 19–24. https://doi.org/10.1145/1462728.1462732
Krumm, J. (2010). Ubiquitous computing fundamentals / edited by John Krumm. CRC Press. http://www.vlebooks.com/vleweb/product/openreader?id=Birmingham&isbn=9781420093612
Salber, D., Dey, A. K., & Abowd, G. D. (1999). The Context Toolkit: Aiding the Development of Context- enabled Applications. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 434–441. https://doi.org/10.1145/302979.303126