Theme 1: Proactive Worlds
(From the TerraSwarm SOW for the last two years) (Aug 6, 2015)
Theme leaders: Dutta and Murray
The goal of this theme is to develop the infrastructural software and architectural techniques for a proactive world, where networked sensors and actuators enable cooperating computing and networking infrastructure to provide smarter environments for humans to operate in. A central part of this effort is a hierarchical and compositional system architecture, supported by a distributed, loosely coupled executive that we call the "SwarmOS." This architecture must accommodate heterogeneous and dynamic compositions of sensor and actuator devices, mobile vehicles, handheld devices, networking components, and cloud infrastructure. A key challenge is to dynamically balance the needs of distributed concurrent application resources, quality of service (QoS), and real-time guarantees. Equally important is the need to respect the privacy and integrity of streams of information. Building on the support of the SwarmOS, TerraSwarm applications will be structured as dynamic, hierarchical graphs of services, with components providing assured quality of service through resource brokerage.
[Jones, Kubiatowitz, Lee, Rabaey, Wawrzynek]
The SwarmOS is the collection of essential services that make swarm applications possible, trustable, robust, and efficient. This task will develop the core components of the SwarmOS that will serve as a distributed executive and resource manager for TerraSwarm applications. The SwarmOS will mediate the needs of applications for services and clusters of resources where resource clusters may be, for example, a portion of a processor's resources, or a slice of bandwidth. The SwarmOS must be distributed, resource aware, governed by service-level contracts, capable of restricting admission and of guaranteeing access to critical services. It must function in a heterogeneous network, where multiple technologies are combined and where connectivity may be disrupted or only available sporadically. The SwarmOS will provide a cross-platform implementation of a QoS-aware communication and archiving service called the global data plane (GDP), location-aware routing and caching services, service discovery, and resource brokerage.
A key goal is to create the “BSD for the Swarm,” an open-source, well documented, thoroughly tested, and bulletproof suite of core SwarmOS utilities. The task includes leveraging efforts to provide an open and free certificate authority for the web, applying similar mechanisms instead to scalable key management in swarm devices. A second direction concerns how to provide sufficient structure and metadata for streams on the GDP to enable effective use of learning techniques. This leverages the extensive experience of the TerraSwarm machine learning team with the structure of large data sets and the formal contracts work of theme 2. We envision self-describing data sets that codify their structure in the language of contracts, and machine learning algorithms that adapt and are able to aggregate even data coming from disparate sources with diverse formats and content. A third direction concerns resource management, specifically resolving contention for shared resources such as network bandwidth. Again, the language of contracts provides a potential formal framework for this work.
[Dutta, Jafari, Kumar, Lee, Pappas, Rowe, Sangiovanni-Vincentelli]
Just as there has been a dramatic build-out of wireless communication infrastructure over the last decade or so, there will be a similar build-out of more capable infrastructure that we call the “immobiles.” These are compute and communication nodes that provide vendor-neutral gateways between local devices and services (accessible via local wireless or wired networks) and a hierarchy of computing and networked services including the cloud. Such an infrastructure extends the cloud by giving it a presence close to the end devices that interact with the physical world (a vision that Cisco calls “The Fog”), and it gives the endpoint devices a generic way to leverage services that reside in the cloud. The SwarmBox is the first prototype of such a component. It has computing and networking capabilities roughly comparable to mobile devices, but unlike mobile devices, the emphasis is not on interaction with humans through touchscreens and audio, and the devices do not (normally) move around. Such an architecture has a number of key potential advantages:
- It enables services that compose devices from multiple vendors.
- It enables controlled quality of service, for example bounding the latency between local sensors and actuators, enabling use of IoT technology for “important” things, which may be safety critical.
- It enables new approaches to security and privacy by enabling interaction between devices without data having to travel to and from data centers.
- It enables robust design that can provide useful networked services even in the presence of network outages, providing better resilience and safety.
The emphasis of the SwarmBox task will be on identifying the software and hardware capabilities and architectures that will make such immobiles most useful. The hardware will be primarily COTS.
[Kubiatowitz, Lee, Rabaey]
This task exposes TerraSwarm capabilities as services, enabling applications to be built by composing these services. Capabilities that could be so exposed include networking, controller synthesis algorithms, sophisticated simulators, machine learning algorithms, databases of public information, etc. One of the key steps is to understand how the characteristics of these swarm services differ from the traditional ones and then show how new system architectures should be designed to effectively address the challenges. For networking, for example, what makes this challenging (especially in the wireless world) is the need for network resource discovery (which may include interference measurement), the creation of scheduling mechanisms so that contract-terms are met, and the development of resource brokerage across networks that may be owned by different administrative domains. Applications that treat networking as a service can then use innovative wireless deployment strategies that combine opportunistic (peer-to-peer) and infrastructure based networking, and support a range of economic trade-off schemes. In all cases, resource discovery, access control (to hardware or information), and contract negotiation mechanisms will be required, in addition to resource management mechanisms and task allocation strategies across scale.
[Blaauw, Dutta, Hartmann, Jafari, Jones, Rabaey, Rosing, Rowe]
A proactive system observes and measures its environment, builds and adapts models of it, estimates context (presence of humans, activities in progress, etc.), and then acts to provide a useful service. For example, in a building security application, a proactive system will unlock a door when, and only when, an authorized person approaches. This task focuses on architectures for converting raw data into context information considering the highly dynamic nature of its surrounding environment, and combining security, privacy, learning, and optimization techniques to build useful applications.
The edge of the swarm will be highly dynamic, with mobile devices and users providing rapidly varying capabilities and demands, with varying and often limited resources such as energy and bandwidth. The TerraSwarm architecture will need to proactively manage their energy use and dynamically reconfigure and optimize the swarm's functionality under varying resource constraints. This task will develop methods for real-time prediction of the availability of energy, computation, and communication resources, and will provide algorithms for trading off these resources against sensor and actuator capabilities. A proactive architecture will adapt to a system as it changes, to meet the needs of the users. This includes dynamic, hierarchical control (to model a realistic, changing environment) that drives actuation decisions and optimization of available (predicted and actual) resources. It also involves a context engine middleware, which employs machine learning and statistical data analysis to translate raw data streams into higher-level abstractions needed by the swarm applications. The system could dynamically determine, for example, whether to execute requested services to the point of service exhaustion, or to conserve energy in order to be able to service future (possibly higher-priority) requests. We will develop a run-time expected-utility-maximization framework that predicts current system capabilities, determines which services can be executed based on whether their utility is sufficient to warrant execution. Based on the hierarchical composability of component subsystems, this new methodology will be applied to successively larger system scales to maximize the overall system utility within energy and bandwidth constraints. The architecture will leverage the domains established through distributed, hierarchical control; the SwarmBoxes for deployment; the SwarmOS’s accessor interface for communication and data access; and the machine learning toolkit for adaptability and prediction of future system needs.
Additionally, we will leverage current ongoing research on the design of state-of-the-art sensors and actuators, including low-energy battery operated devices, energy-scavenging devices, and wearable devices, by developing common service-based interfaces. These interfaces will be used by TerraSwarm applications and the SwarmOS to provide consistent and reliable service despite the underlying variability of the devices and resources available at any given place and time. Resource brokers will manage limited resources and proactively optimize resource-constrained expected utility. The interfaces will enable mobile devices to interact opportunistically with sensors and actuators in their local area to provide local "eyes, ears, hands, and feet" supporting total situational awareness.
[Bilmes, Blaauw, Dutta, Jafari, Jones, Lee, Murray, Pappas, Rosing, Rowe]
This task focuses on a visionary and long-term challenge problem that serves to drive and test technology development in TerraSwarm. Specifically, the high-level goal is to show that "urban health" can be monitored and predicted through large scale deployments of sensors, both mobile and fixed; strategic placement of “immobiles” providing localized cloud-like services; machine learning for context formation and anomaly detection; optimization for maximizing information extracted from data; and adaptation to changing resources, including opportunistic exploitation of mobiles and wearables, device failures and additions, and deployable mobile vehicles carrying networking and SwarmBox services. Here “urban health” broadly means forming context information that can be used to predict infrastructure failures, sudden changes in the urban environment, unusual citizen activity, or opportunities for revitalization. It includes modeling the behavior of people using larger datasets from population studies to understand better how people interact and affect swarm dynamics at an urban scale. Aspects of the problem include at least:
- Building context awareness from limited and noisy sensor data.
- Anomaly detection.
- Distribution of sensors to allow for understanding stress and rebirth of a city.
- Control of mobile sensors and placement of immobile sensors.
- Adaptive movement of tasks among mobiles, immobiles, and the cloud, as determined by context.
- Design of system architectures that scale to address swarm size problems.
- Obtaining location and proximity information.
- DoD applications to nation building, allowing for macroscale understanding of normal city dynamics.
- Civilian applications to urban renewal in stressed cities such as Detroit.
- Interactions between the society behavior/habits and the swarm of devices.
- Use of sensors in "resource starved" environments (like third world cities) which are not providing rich data sets for higher level analytics.
- Preserving privacy, for example by leveraging the SwarmBox architecture to keep sensor data local by default, but publish only summary statistics.
There is a rich set of possible sensing modalities that can contribute to such urban context formation, including collecting acoustic, seismic, air quality, image, and chemical data. Consider for example acoustic data. Sound can help characterize many aspects of a city: traffic; rhythm; daily, weekly, and yearly cycles; curfews; estimates of the number of people on the streets at what times and neighborhoods, and perhaps their "mood" from the aggregate conversation patterns; and time, locations, and amount of gunfire, altercations, etc.
In addition to DoD and urban revival applications, there are applications to health. The long-term negative health impacts of urban noise are beginning to be quantified, and the results are disturbing; a large-scale World Health Organization study in Europe with solid methodology estimated a collective average loss of 3-4 months of lifespan of Europeans due (largely) to increased low-level-stress-related heart-attack risk induced by environmental noise. Large-scale noise monitoring is extremely expensive today, so it’s very rarely done, but particularly when coupled with the vision of the deployment of the “immobiles,” it starts to look viable and cost effective. Similarly, medical studies have shown that frequent exposure to air pollutants dramatically increases incidence of asthma, one of the top chronic conditions in the nation. Leveraging both mobile and immobile systems to understand how individual characteristics (genetics, personal history etc), daily choices and exposure to pollutants over time contribute to development of asthma. Another possible angle on this project is to opportunistically leverage mobiles and wearables, the latter of which are expected to greatly increase in number in the next few years. Specifically, a wealth of information can be extracted from sensors carried on these devices, but the challenge is to translate the raw data into context awareness. What does it mean when many proximate heartbeats are elevated, for example? Data analytics and machine learning will play a critical role.
- Urban Heartbeat (wiki)
In the face of the huge explosion of interest in the Internet of Things, together with high-profile security failures in Information Technology, a major risk factor for deployment and impact of swarm technology is developing and maintaining trust among the general public. Issues of trust, privacy and security will be key to the acceptance and adoption of these technologies. This task puts a focus on the interactions that humans and culture have with this emerging technology. There are technological and humanistic aspects of this problem, and the two are somewhat intertwined. For instance, understandable and reliable swarm programs could go a long way towards fostering trust. Users need to be able to control where their data goes, and authentication and encryption should be routine and visible by default, rather than requiring a special effort on top of other development efforts. This task covers new methods and design patterns that make security and privacy implications of the technology more visible to users. Trust is also be undermined if systems are unreliable or unpredictable, exhibiting unexpected behavior. So this task includes efforts to understand what makes one way of swarm composition more error-prone, than another.
Task 1.7: The Human Intranet - Bridging the Human-World Information Gap
[Rabaey, Arias, Abbeel, Carmena, Hartmann, Maharbiz]
(From the Human Intranet Proposal, October, 2015)
With the explosive growth of the ”smart” society, enormous amounts of information are instantaneously available in the enhanced world around us, or the cyberworld beyond. Hence one may wonder if the traditional human input/output modalities have the necessary bandwidth or expressiveness to effectively deal with the increasing pace of an “augmented world”. The questions we are trying to address is if and how advances in semiconductor and information processing technologies may help to alter this unbalance?
A first-generation Human Intranet intended to boost human input-output performance will be developed. Given the limited two-year span, we plan to focus on just a couple of use cases. One would focus on hybrid sensory expansion, while another would enable higher information output throughput through low-SNR actuation devices (such as pressure sensitive on-skin keyboards). Prototype systems will be built using existing as well as emerging devices. These prototypes will be used to evaluate the effectiveness of the techniques in terms of measurable parameters such as throughout increase, sensitivity enhancement, scope extension and learning complexity.
- Richard Murray (Leader)
- Prabal Dutta (Co-Leader)
- See the Services Overview for a list of other faculty.
Proactive Worlds Resources
You are not logged in or do not have access to most of this workgroup. Admins, to edit this page, see the profile.