A Practical Introduction to Human-in-the-loop Cyber-physical Systems

The first book focusing on one of the hottest new topics in Internet of Things systems research and development

Studies estimate that by 2020 we will have a vast Internet of Things (IoT) network comprising 26 billion connected devices, including everything from light bulbs to refrigerators, coffee makers to cars. From the beginning, the concept of cyber-physical systems (CPS), or the sensing and control of physical phenomena through networks of devices that work together to achieve common goals, has been implicit in the IoT enterprise. This book focuses on the increasingly hot topic of Human-in-the-loop Cyber-Physical Systems (HiTLCPS)—CPSs that incorporate human responses in IoT equation.

Why have we not yet integrated the human component into CPSs? What are the major challenges to achieving HiTLCPS? How can we take advantage of ubiquitous sensing platforms, such as smartphones and personal devices to achieve that goal? While mature HiTLCPS designs have yet to be achieved, or a general consensus reached on underlying HiTLCPS requirements, principles, and theory, researchers and developers worldwide are on the cusp of realizing them. With contributions from researchers at the cutting edge of HiTLCPS R&D, this book addresses many of these questions from the theoretical and practical points of view.

• An essential primer on a rapidly emerging Internet-of-Things concept, focusing on human-centric applications
• Discusses new topics which, until now, have only been available in research papers scattered throughout the world literature
• Addressed fundamental concepts in depth while providing practical insights into the development of complete HiTLCPS systems
• Includes a companion website containing full source-code for all of the applications described

This book is an indispensable resource for researchers and app developers eager to explore HiTL concepts and include them into their designs. It is also an excellent primer for advanced undergraduates and graduate students studying IoT, CPS, and HiTLCPS.

David Nunes, PhD is an information systems advisor and software developer at the Central Portuguese Bank. He holds a Masters degree in Bioinformatics, and a PhD in Informatics, both from the University of Coimbra, Portugal.

Jorge Sá Silva, PhD is an Assistant Professor in the Department of Informatics Engineering at the tUniversity of Coimbra, Portugal.

Fernando Boavida, PhD is a Professor in the Department of Informatics Engineering at the University of Coimbra and founder of the University's Laboratory of Communications and Telematics of DEI.

List of Figures xi

List of Tables xvii

Foreword xix

Preface xxi

Acknowledgments xxiii

List of Abbreviations xxv

About the CompanionWebsite xxvii

1 Introduction 1

1.1 The Rise of Cyber-Physical Systems 1

1.2 Humans as Elements of Cyber-Physical Systems 4

1.3 Objectives and Structure 7

Part I Evolution and Theory 9

2 Evolution of HiTL Technologies 11

2.1 "Things", Sensors, and the RealWorld 11

2.2 Human Sensing and Virtual Communities 17

2.3 In Summary... 23

3 TheoryofHiTLCPSs 25

3.1 Taxonomies for HiTLCPSs 25

3.2 Data Acquisition 28

3.2.1 Humans as Sets of Sensors 28

3.2.2 Humans as Communication Nodes 29

3.3 State Inference 30

3.3.1 Human Nature 30

3.3.2 Humans as Processing Nodes 31

3.4 Actuation 31

3.4.1 Humans and Robots as Actuators 31

3.5 In Summary... 32

4 HITL Technologies and Applications 35

4.1 Technologies for Supporting HiTLCPS 35

4.1.1 Data Acquisition 35

4.1.2 State Inference 39

4.1.3 Actuation 42

4.2 Experimental Projects 45

4.2.1 HiTL in Industry and at Home 46

4.2.2 HiTL in Healthcare 48

4.2.3 HiTL in Smartphones and Social Networking 51

4.3 In Summary... 56

Part II Human-in-the-Loop: Hands-On 57

5 ASampleApp 59

5.1 A Sample Behavior Change Intervention App 59

5.2 The Sample App’s Base Architecture 60

5.2.1 The Android App 60

5.2.2 The Server 65

5.3 Enhancing the Sample App with HiTL Emotion-awareness 66

5.3.1 Choosing a Machine Learning Technique 67

5.3.2 Implementing Emotion-awareness 68

5.4 In Summary... 71

6 Setting up the Development Environment 73

6.1 Installing Android Studio 73

6.2 Cloning the Android Project 77

6.3 Deploying the Server 85

6.3.1 Installing the Software and Cloning the Server’s Project 85

6.3.2 Obtaining a Foursquare®’s Client ID and Client Secret 88

6.3.3 Setting up the Database 91

6.3.4 Deploying the Server on Tomcat 7 95

6.4 Testing the Sample App 99

6.5 In Summary... 101

7 Data Acquisition 103

7.1 Creating the EmotionTasker 103

7.2 Processing Sensory Data 110

7.3 In Summary... 118

8 StateInference 121

8.1 Implementing a Neural Network 121

8.2 Requesting User Feedback 125

8.2.1 Creating the EmotionFeedback Activity 125

8.2.2 Implementing the EmotionSpace View 127

8.2.3 Finishing EmotionFeedback 151

8.2.4 Showing a Feedback Request Notification 158

8.3 Processing User Feedback 168

8.3.1 Processing Feedback on the EmotionTasker 168

8.3.2 Training the Neural Network 170

8.3.3 Sending Emotional Information to the Server 173

8.4 In Summary... 176

9 Actuation 179

9.1 Handling Emotions on the Server 179

9.1.1 Parsing JSON Requests 180

9.1.2 Creating theWeb Interface 189

9.1.3 Creating the Server’s BackgroundThread 190

9.1.4 Processing Incoming Emotions 192

9.1.5 Pruning Outdated Emotions 208

9.2 Finishing up EmotionTasker 210

9.2.1 Handling ANN Output 212

9.2.2 Posting New Emotion Inferences 217

9.3 Providing Positive Reinforcement 226

9.3.1 Creating a Motivational Dialog Box 226

9.3.2 Enabling the Emotion Heatmaps 234

9.4 In Summary… 237

Part III Future of Human-In-the-Loop Cyber-Physical Systems 239

10 Requirements and Challenges for HiTL Applications 241

10.1 Resilience 241

10.2 Security and Privacy 242

10.3 Standard Communications 244

10.4 Localization 248

10.5 State Inference 248

10.6 Safety 250

10.7 In Summary… 250

11 Human-in-the-Loop Constraints 253

11.1 Technical Limitations 253

11.2 Ethical limitations 256

A EmotionTasker’s full code 261

References 275

Index 289

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.