The Defense Advanced Research Projects Agency (DARPA) is looking to fund research into shallow neural network architectures that could run accurately on low-powered Edge systems.

Hyper-Dimensional Data Enabled Neural Networks (HyDDENN) would be able to provide similar results compared to existing state-of-the-art (SOA) Deep Neural Networks (DNN) running in hyperscale data centers, but without the latency and large computational requirements.

Smaller and faster

Conventional DDNs are "growing wider and deeper, with the complexity growing from millions to hundreds of millions of parameters in the last few years," a DARPA presolicitation document says. "The basic computational primitive to execute training and inference functions in DNN is the multiply and accumulate (MAC) operation. As DNN parameter count increases, SOA networks require tens of billions of MAC operations to carry out one inference."

This means that the accuracy of DNN "is fundamentally limited by available MAC resources," DARPA says. "Consequently, SOA high accuracy DNNs are hosted in the cloud centers with clusters of energy hungry processors to speed up processing. This compute paradigm will not satisfy many DoD applications which demand extremely low latency, high accuracy artificial intelligence (AI) under severe size, weight, and power constraints."

With HyDDENN, the agency hopes to break free from a reliance on large MAC-based DDNs. "HyDDENN will explore and develop innovative data representations with shallow NN architectures based on efficient, non-MAC, digital compute primitives to enable highly accurate and energy efficient AI for DoD Edge systems."

The aim is to reduce parameter counts by at least 10x, while maintaining accuracy in comparison with a similar MAC-based DNN solution. "With efficient digital compute hardware, these innovations will lead to at least 100x reduction in combined compute power and throughput, while retaining high-accuracy output when compared to the SOA DNN approach."

Although DARPA's focus is on military applications - where the increasingly-connected battlefield will require significant tactical Edge deployments - the agency believes the technology could find use elsewhere.

"It is expected that HyDDENN will have significant impact in the areas of Edge/IoT communications and contextual Edge sensing and classification," the document states. DoD-relevant applications mentioned by DARPA include contextual communications, speech recognition, gesture recognition, and medical diagnostics.

For a project with such lofty aims, HyDDENN has limited funding available. The award value of the Phase 1 Feasibility Study (6 months) tops out at $300,000, while the Phase 2 Proof of Concept (12 months) should not exceed $700,000.

By the end of the second phase, the project's researchers are expected to have developed an ASIC architecture and high level logic designs at register-transfer level "as well as a project plan to implement a future fully programmable integrated chip-scale digital IC with the proposed HD data representation, logic primitives, and shallow HD NN to attain the HyDDENN performance goals and metrics for the targeted application."

Proposals to HyDDENN are due by December 11, 2019, at 12:00 PM ET. "For planning and budgetary purposes, proposers should assume a program start date of February 12, 2020."

Project documents appear to suggest HyDDENN is headed by program manager Dr. Young-Kai Chen, who previously worked as a senior director at Bell Laboratories.

Among his achievements is leading the "exploration of coherent optics with advanced digital signal processing algorithms, which resulted in the launch of the first commercial 100G coherent optical receivers in 2011," his DARPA bio states.

"Through this technology, large data centers can be linked thousands of kilometers across continents without electronic repeaters."