Methods and Techniques in Deep Learning Advancements in mmWave Radar Solutions

Introduces multiple state-of-the-art deep learning architectures for mmWave radar in a variety of advanced applications

Methods and Techniques in Deep Learning: Advancements in mmWave Radar Solutions provides a timely and authoritative overview of the use of artificial intelligence (AI)-based processing for various mmWave radar applications. Focusing on practical deep learning techniques, this comprehensive volume explains the fundamentals of deep learning, reviews cutting-edge deep metric learning techniques, describes different typologies of reinforcement learning (RL) algorithms, highlights how domain adaptation (DA) can be used for improving the performance of machine learning (ML) algorithms, and more. Throughout the book, readers are exposed to product-ready deep learning solutions while learning skills that are relevant for building any industrial-grade, sensor-based deep learning solution.

A team of authors with more than 70 filed patents and 100 published papers on AI and sensor processing illustrates how deep learning is enabling a range of advanced industrial, consumer, and automotive applications of mmWave radars. In-depth chapters cover topics including multi-modal deep learning approaches, the elemental blocks required to formulate Bayesian deep learning, how domain adaptation (DA) can be used for improving the performance of machine learning algorithms, and geometric deep learning are used for processing point clouds. In addition, the book:

• Discusses various advanced applications and how their respective challenges have been addressed using different deep learning architectures and algorithms
• Describes deep learning in the context of computer vision, natural language processing, sensor processing, and mmWave radar sensors
• Demonstrates how deep parametric learning reduces the number of trainable parameters and improves the data flow
• Presents several human-machine interface (HMI) applications such as gesture recognition, human activity classification, human localization and tracking, in-cabin automotive occupancy sensing

Methods and Techniques in Deep Learning: Advancements in mmWave Radar Solutions is an invaluable resource for industry professionals, researchers, and graduate students working in systems engineering, signal processing, sensors, data science, and AI.

Avik Santra is Head of Advanced Artificial Intelligence at Infineon Technologies, Munich, Germany.

Souvik Hazra is a Senior Staff Machine Learning Engineer at Infineon Technologies, Munich, Germany.

Lorenzo Servadei is a Senior Staff Machine Learning Engineer at Infineon Technologies and a Lecturer at The Technical University of Munich (TU München), Germany.

Thomas Stadelmayer is a Machine Learning Engineer at Infineon Technologies, Munich, Germany.

Michael Stephan is a PhD candidate employed at Infineon Technologies, Munich, Germany.

Anand Dubey is a Staff Machine Learning Engineer at Infineon Technologies and a Research Assistant at Friedrich-Alexander-University of Erlangen-Nürnberg, Germany

Preface

Acronyms

1 Introduction to Radar Processing & Deep Learning 1

1.1 Basics of Radar Systems 1

1.1.1 Fundamentals 2

1.1.2 Signal Modulation 2

1.2 FMCW Signal Processing 6

1.2.1 Frequency-Domain Analysis 7

1.3 Target Detection & Clustering 14

1.4 Target Tracking 19

1.4.1 Track Management 21

1.4.2 Track Filtering 22

1.5 Target Representation 28

1.5.1 Image Representation 30

1.5.2 Point-Cloud Maps 34

1.6 Target Recognition 36

1.6.1 Feedforward Network 37

1.6.2 Convolutional Neural Networks (CNN) 37

1.6.3 Recurrent Neural Network (RNN) 43

1.6.4 Autoencoder & Variational Autoencoder 47

1.6.5 Generative Adversial Network 51

1.6.6 Transformer 54

1.7 Training a Neural Network 56

1.7.1 Forward Pass & Backpropagation 57

1.7.2 Optimizers 62

1.7.3 Loss Functions 65

1.8 Questions to the Reader 66

Bibliography 68

2 Deep Metric Learning 75

2.1 Introduction 78

2.2 Pairwise methods 79

2.2.1 Contrastive Loss 79

2.2.2 Triplet Loss 80

2.2.3 Quadruplet Loss 81

2.2.4 N-Pair Loss 82

2.2.5 Big Picture 83

2.3 End-to-end Learning 84

2.3.1 Cosine Similarity 86

2.3.2 Euclidean Distance 95

2.3.3 Big Picture 100

2.4 Proxy methods 103

2.5 Advanced Methods 103

2.5.1 Statistical Distance 104

2.5.2 Structured Metric Learning 108

2.6 Application Gesture Sensing 110

2.6.1 Radar System Design 111

2.6.2 Data Set and Preparation 112

2.6.3 Architecture and Metric Learning Procedure 114

2.6.4 Results 123

2.7 Questions to the Reader 129

Bibliography 130

3 Deep Parametric Learning 135

3.1 Introduction 135

3.2 Radar Parametric Neural Network 140

3.2.1 2D Sinc Filters 142

3.2.2 2D Morlet Wavelets 143

3.2.3 Adaptive 2D Sinc Filters 145

3.2.4 Complex Frequency Extraction Layer 146

3.3 Multilevel Wavelet Decomposition Network 150

3.4 Application Activity Classification 153

3.4.1 Proposed Parametric Networks 155

3.4.2 State-of-art Networks 158

3.4.3 Results & Discussion 160

3.5 Conclusion 167

3.6 Question to Readers 168

Bibliography 168

4 Deep Reinforcement Learning 173

4.1 Useful Notation and Equations 173

4.1.1 Markov Decision Process 173

4.1.2 Solving the Markov Decision Process 174

4.1.3 Bellman Equations 175

4.2 Introduction 175

4.3 On-Policy Reinforcement Learning 179

4.4 Off-Policy Reinforcement Learning 180

4.5 Model-Based Reinforcement Learning 180

4.6 Model-Free Reinforcement Learning 181

4.7 Value-Based Reinforcement Learning 181

4.8 Policy-Based Reinforcement Learning 183

4.9 Online Reinforcement Learning 183

4.10 Offline Reinforcement Learning 184

4.11 Reinforcement Learning with

Discrete Actions 184

4.12 Reinforcement Learning with

Continuous Actions 185

4.13 Reinforcement Learning Algorithms

for Radar Applications 185

4.14 Application Tracker’s Parameter Optimization 189

4.14.1 Motivation 190

4.14.2 Background 192

4.14.3 Approach 202

4.14.4 Experimental 208

4.14.5 Outcomes of the proposed Approach 219

4.15 Conclusion 220

4.16 Questions to the Reader 220

Bibliography 221

5 Cross-Modal Learning 229

5.1 Introduction 229

5.2 Self-Supervised Multi-Modal Learning 233

5.2.1 Generating Audio Statistics 233

5.2.2 Predicting sounds from images 234

5.2.3 Audio Features Clustering 234

5.2.4 Binary Coding Model 235

5.2.5 Training 235

5.2.6 Results 235

5.3 Joint Embeddings Learning 237

5.3.1 Feature Representations 237

5.3.2 Joint-Embedding Learning 238

5.3.3 Matching & Ranking 239

5.3.4 Training Details & Result 239

5.3.5 Discussion 241

5.4 Multi-Modal Input 241

5.4.1 Multi-modal Compact Bilinear Pooling 242

5.4.2 VQA Architecture 243

5.4.3 Training Details & Result 245

5.4.4 Discussion 245

5.5 Cross-Modal Learning 245

5.5.1 Data Acquisition 246

5.5.2 Cross-Modal Learning for Key-Point Detection 246

5.5.3 Training Details & Result 247

5.5.4 Discussion 249

5.6 Application People Counting 250

5.6.1 FMCW Radar System Design 251

5.6.2 Data Acquisition 252

5.6.3 Solution 1 253

5.6.4 Solution 2 262

5.7 Conclusion 265

5.8 Questions to the Reader 265

Bibliography 267

6 Signal Processing with Deep Learning 273

6.1 Introduction 273

6.2 Algorithm Unrolling 274

6.2.1 Learning Fast Approximations of Sparse Coding 275

6.2.2 Learned ISTA in radar processing 279

6.3 Physics-inspired Deep Learning 282

6.4 Processing-specific Network Architectures 284

6.5 Deep Learning-aided Signal Processing 288

6.6 Questions to the Reader 297

Bibliography 297

7 Domain Adaptation 303

7.1 Introduction 303

7.2 Transfer Learning and Domain Adaptaton 304

7.3 Categories of Domain Adaptation 307

7.3.1 Common Data Shifts 307

7.3.2 Methods of Domain Adaptation 308

7.4 Domain Adaptation in Radar Processing 315

7.4.1 Domain Adaptation with a different Sensor Type 316

7.4.2 Domain Adaptation with different Radar Settings 318

7.5 Summary 331

7.6 Questions to the Reader 331

Bibliography 332

8 Bayesian Deep Learning 339

8.1 Learning Theory 341

8.2 Bayesian Learning 343

8.3 Bayesian Approximations 352

8.4 Application VRU Classification 372

8.4.1 VAE as Bayesian 373

xiii

8.4.2 Bayesian Metric Learning 377

8.4.3 Kalman as Bayesian 383

8.4.4 Results 387

8.5 Summary 391

8.6 Questions to the Reader 393

Bibliography 393

9 Geometric Deep Learning 397

9.1 Representation Learning in Graph Neural Network 399

9.1.1 Fundamentals 399

9.1.2 Learning Theory 401

9.1.3 Embedding Learning 406

9.2 Graph Representation Learning 407

9.2.1 Convolution GNN 408

9.2.2 Recurrent Graph Neural Networks (RGNN) 409

9.2.3 Graph Autoencoders (GAE) 409

9.2.4 Spatial–Temporal Graph Neural Networks (STGNN) 410

9.2.5 Attention GNN 410

9.2.6 Message-passing GNN 411

9.3 Applications 413

9.3.1 Application 1 Long-Range Gesture Recognition 413

9.3.2 Application 2 Bayesian Anchor-Free Target Detection 426

9.4 Conclusion 444

9.5 Questions to the Reader 445

Bibliography 446

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.