Patent Drafting Analysis of DeepMind Technologies’ Augmented Recurrent Neural Network with External Memory | US 12,099,928 B2

IP Drafting Analysis · US 12,099,928 B2

Patent Drafting Analysis of DeepMind Technologies' Augmented Recurrent Neural Network with External Memory | US 12,099,928 B2

Q: What does US 12,099,928 B2 protect?

US 12,099,928 B2 protects augmented recurrent neural network systems, methods, and computer-readable media that couple a neural network to a differentiable external matrix memory governed by a size vector. The patent specifically claims the mechanism by which a memory interface subsystem determines pop, push, and write parameters from the neural network output to update the external memory's matrix rows and size vector entries, then reads a weighted sum of matrix rows back into the neural network input — enabling the neural network to implement continuously differentiable analogues of stack, queue, and double-ended queue data structures with logically unbounded memory capacity.

Q: Who owns patent US 12,099,928 B2?

US 12,099,928 B2 is owned by DeepMind Technologies Limited, headquartered in London, United Kingdom. The inventors are Edward Thomas Grefenstette (London, GB), Karl Moritz Hermann (Berlin, DE), Mustafa Suleyman (London, GB), and Philip Blunsom (Oxford, GB).

Q: What are the independent claims of US 12,099,928 B2?

Claim 1 is a system/apparatus claim directed to an augmented neural network system comprising a neural network and a memory interface subsystem that maintains a matrix memory with a size vector to enable differentiable read and write operations on external memory. Claim 12 is a method claim covering the same operational steps as a process for generating system outputs using the augmented neural network system. Claim 20 is a computer-readable media (CRM) claim covering non-transitory storage media encoded with instructions to implement the same augmented neural network system as Claim 1.

Q: What technology does this patent cover?

This patent covers a way to give artificial neural networks access to an external memory bank that can grow or shrink dynamically — like giving a computer program both short-term working memory (inside the neural network) and long-term storage (in the external memory). The key innovation is that the memory can be read and written using smooth mathematical operations (not binary on/off switches), which means the entire system — including how it uses memory — can be trained automatically using standard machine learning techniques. This allows neural networks to handle complex tasks requiring long-range memory, like language translation, logical reasoning, or sequence processing, far more effectively than traditional neural networks operating on fixed-size internal states alone.

Q: What IPC classifications does this patent use?

G06N 3/08 (2023.01) — Artificial neural networks: learning. G06N 3/044 (2023.01) — Artificial neural networks: recurrent networks. G06N 3/063 (2023.01) — Artificial neural networks: physical realisation. G06N 3/082 (2023.01) — Artificial neural networks: hybrid architectures.

A structural and strategic analysis of DeepMind's granted patent covering augmented RNNs with differentiable external memory, examining claim architecture, drafting quality, prosecution defensibility, and critical coverage gaps.

US 12,099,928 B2Filed: Feb 24, 2023Granted: Sep 24, 2024G06N 3/08G06N 3/044G06N 3/063G06N 3/082

Draft a Patent Like This Explore Related Patents

Spec Words

6,200

Across 6 sections

Draft now ↗

Total Claims

3 independent · 17 dependent

Draft now ↗

Figure Sheets

System architecture, process flow diagrams

Draft now ↗

Published by PatSnap Insights Team · June 4, 2026 · 12 min read Verified by PatSnap Eureka Data

Overview

Structural Overview

The detailed description dominates at approximately 50% of total specification words (~3,100 of ~6,200), with the claims section representing a substantial ~28% — reflecting a claim-heavy continuation filing strategy rather than an exploratory disclosure. The patent contains exactly 20 claims structured across 3 independent claims (Claims 1, 12, and 20) covering system, method, and CRM formats, with 17 dependent claims providing layered refinements of the stack, queue, and double-ended queue memory embodiments. The 5 drawing sheets are exclusively process flow diagrams and a high-level system architecture, providing functional but not circuit-level figure coverage.

Section Word Distribution

↗ Click bars to explore

Figure Inventory — 5 Sheets

Figure	Description	Role
FIG. 1	High-level architecture of the Augmented Neural Network System 100 showing the Recurrent Neural Network 102, Memory Interface Subsystem 106, and External Memory 104 with read (r), write (w), hidden state (h), system input (i), and system output (o) signal flows.Search in Eureka ↗	System architecture
FIG. 2	Flow diagram of process 200 for generating a system output from a system input, covering steps 202–216 including receiving input, obtaining memory data, generating neural network input, processing via RNN, generating system output, determining memory state parameters, updating external memory, and reading updated memory.Search in Eureka ↗	Flow diagram
FIG. 3	Flow diagram of process 300 for updating and reading from external memory when configured as a continuous stack memory, showing steps 302–310 covering determination of pop value, push value, and write vector, followed by memory write, size vector update via pop, push addition, and memory read.Search in Eureka ↗	Claim support
FIG. 4	Flow diagram of process 400 for updating and reading from external memory when configured as a continuous queue memory, structurally parallel to FIG. 3 but using a pull value (step 406) instead of a pop value to traverse the size vector from lowest to highest entry.Search in Eureka ↗	Claim support
FIG. 5	Flow diagram of process 500 for updating and reading from external memory when configured as a continuous double-ended queue memory, showing steps 502–510 with separate top and bottom pop/push values and write vectors, yielding both a top read vector and a bottom read vector from external memory.Search in Eureka ↗	Claim support

Analysis powered by PatSnap Eureka. Patent text and figures publicly available from USPTO. Draft a Similar Patent

Claims

Claim Architecture Analysis

The patent contains exactly 3 independent claims — Claim 1 (system/apparatus), Claim 12 (method), and Claim 20 (computer-readable media/CRM) — providing full tripartite enforcement coverage. The dependent:independent ratio of 5.67:1 is slightly below the typical 6–10:1 norm for the G06N neural network IPC class, suggesting moderate but not exhaustive dependent claim fallback. The structural parallelism across Claims 1, 12, and 20 is a deliberate strategy enabling enforcement against hardware manufacturers, software operators, and storage medium distributors respectively, though the parallel structure means many dependent claims (e.g., Claims 2–11, 13–19) merely replicate each other's technical refinements across the three formats rather than introducing genuinely distinct limitations.

Core inventive concept: The claims address the problem of bounded memory in standard recurrent neural networks by coupling a neural network to a differentiable external matrix memory governed by a size vector — enabling the RNN to implement continuously differentiable analogues of stack, queue, and double-ended queue data structures via learned pop, push, and write parameters. Claim 1's memory interface subsystem performs the critical operations: determining memory state parameters from the neural network output, writing data to new rows of the matrix memory, updating the size vector, reading a weighted sum of matrix rows using size-vector-derived weights, and feeding the result back as the next neural network input.

Independent Claim Dissection

Claim	Preamble	Transition	Key Body Elements
Claim 1	An augmented neural network system for processing one or more inputs to generate one or more system outputs, the augmented neural network system	comprising:	a neural network configured to receive a first network input and generate a neural network output; a memory interface subsystem configured to: provide output derived from neural network output as system output, determine memory state parameters from neural network output, update current state of external memory (matrix memory + size vector) using memory state parameters, read data from external memory using weighted sum of matrix rows per updated size vector entries, combine read data with system input to generate second neural network inputSearch prior art ↗
Claim 12	A method for processing one or more inputs to generate a sequence of system outputs using an augmented neural network system comprising a neural network,	wherein the method comprises:	providing output derived from neural network output as system output, determining memory state parameters from neural network output, updating current state of external memory (matrix memory + size vector) using memory state parameters (write new rows, update size vector), reading data from external memory via weighted sum of matrix rows per updated size vector, combining read data with system input to generate second neural network inputSearch prior art ↗
Claim 20	One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to implement an augmented neural network system for processing one or more inputs to generate one or more system outputs, the augmented neural network system	comprising:	a neural network configured to receive first network input and generate neural network output; a memory interface subsystem configured to: provide output derived from neural network output as system output, determine memory state parameters, update current state of external memory (matrix memory + size vector) via writing new rows and updating size vector, read data via weighted sum of matrix rows per size vector weights, combine read data with system input to generate second neural network inputSearch prior art ↗

Claim Dependency Tree

1 System claim — augmented neural network with external matrix memory and size-vector-based read/write interface subsystemSearch Claim 1 prior art ↗

2 Adds: determining memory state parameters comprises determining pop value, push value, and write vector from neural network outputSearch in Eureka ↗

3 Adds: updating current state comprises adding write vector as new row of matrixSearch in Eureka ↗

4 Adds: updating size vector comprises updating entries using pop value as weights and adding push value as new entrySearch in Eureka ↗

5 Adds: respective weights are the corresponding entries of the size vectorSearch in Eureka ↗

6 Adds: combining read data with system input comprises concatenating read vector and system inputSearch in Eureka ↗

7 Adds: determining memory state parameters comprises determining top pop value, bottom pop value, top push value, bottom push value, top write vector, bottom write vector (deque embodiment)Search in Eureka ↗

8 Adds: updating current state comprises adding top write vector as new top row and bottom write vector as new bottom row of matrixSearch in Eureka ↗

9 Adds: updating current state comprises two-phase size vector update using top pop for initial update then bottom pop for final update, plus adding top and bottom push valuesSearch in Eureka ↗

10 Adds: reading data comprises reading both a top read vector (highest-to-lowest traversal) and a bottom read vector (lowest-to-highest traversal) from external memorySearch in Eureka ↗

11 Adds: combining data comprises concatenating top read vector, bottom read vector, and system input to generate second neural network inputSearch in Eureka ↗

12 Method claim — parallel structure to Claim 1 covering method steps for augmented neural network with external matrix memory and size-vector-based read/writeSearch Claim 12 prior art ↗

13 Adds: determining memory state parameters comprises determining pop value, push value, and write vectorSearch in Eureka ↗

14 Adds: updating current state comprises adding write vector as new row of matrixSearch in Eureka ↗

15 Adds: updating size vector comprises updating entries using pop value and adding push value as new entrySearch in Eureka ↗

16 Adds: respective weights are the corresponding entries of the size vectorSearch in Eureka ↗

17 Adds: combining data comprises concatenating read vector and system inputSearch in Eureka ↗

18 Adds: determining memory state parameters comprises determining top pop, bottom pop, top push, bottom push, top write vector, bottom write vector (deque embodiment)Search in Eureka ↗

19 Adds: updating current state comprises adding top write vector as new top row and bottom write vector as new bottom rowSearch in Eureka ↗

20 CRM claim — non-transitory computer-readable storage media encoding instructions to implement augmented neural network system with external matrix memory and size-vector interfaceSearch Claim 20 prior art ↗

Metric	This Application	Software / AI Industry Norm
Total claims	20	15 – 25
Independent claim count	3	2 – 4
Dependent : Independent ratio	5.67 : 1	5 – 9 : 1
Method claims present?	Yes — Claim 12	Common
System / apparatus claims?	Yes — Claim 1	Common

Analysis powered by PatSnap Eureka. Patent text and figures publicly available from USPTO. Draft a Similar Patent

Drafting Quality

Drafting Quality Signals

The claim set demonstrates strong tripartite structural coverage (Claims 1, 12, 20) and clear antecedent basis management across technically dense claim language involving matrix memories and size vectors. The primary drafting weakness lies in the high degree of structural mirroring between the three independent claim chains — the dependent claims for Claims 12 and 20 (Claims 13–19) essentially replicate the limitations of Claims 2–11 without introducing genuinely independent fallback positions, reducing the effective claim diversity.

✅

Antecedent Basis

Antecedent basis is clean throughout the 20-claim set. Claim 1 introduces "a neural network" and "a memory interface subsystem" in the preamble body, and all subsequent references use "the neural network" and "the memory interface subsystem" consistently. "The external memory," "the matrix memory," "the size vector," "the neural network output," and "the system input" all have proper antecedent introductions within Claim 1 before being referenced in dependent Claims 2–11. Claims 12 and 20 repeat this structure correctly.

Explore in Eureka ↗

✅

Spec–Claim Consistency

Specification support for the independent claim limitations is robust. The matrix memory structure and size vector concept (Claim 1) map directly to the detailed description at columns 3–4 with the formula V(i,j) and size vector s. The read operation (weighted sum of matrix rows) is supported by the description of the memory interface subsystem at column 4 and FIG. 2 step 216. The write operation (adding write vector as new row) is supported by FIG. 3 step 304. The combination operation for generating the second neural network input (Claim 1's final limitation) maps to FIG. 2 step 206 and column 4 description of concatenation.

Explore in Eureka ↗

✅

Transition Word Usage

All three independent claims use "comprising" as the transition, which is the strategically optimal choice for this technology — it allows the augmented system to include additional components (e.g., attention mechanisms, transformer layers) without escaping claim scope. The memory interface subsystem's operations also use "comprising" for their sub-limitations, correctly allowing for additional operations beyond those explicitly recited. No missed opportunities for "consisting essentially of" are apparent, as the open-ended scope is appropriate for a foundational architecture patent.

Explore in Eureka ↗

⚠️

§112(f) Means-Plus-Function Risk

Claim 1 recites "a memory interface subsystem" defined entirely by a functional list of operations it is "configured to" perform — this is the modern functional claiming pattern that, while technically avoiding literal "means for" language, can attract §112(f) scrutiny if an examiner argues that "subsystem" is a non-structural term. The specification does not define the memory interface subsystem in structural terms beyond its functional role (column 3, lines 15–20), providing no clear structural definition to rebut a §112(f) challenge. A stronger filing would have included at least one structural definition (e.g., processor, memory controller, or circuit elements) within the memory interface subsystem description.

Explore in Eureka ↗

⚠️

§101 Eligibility Risk

Claims 1, 12, and 20 present moderate Alice exposure because the core innovation — differentiable external memory management with pop/push/write operations — is fundamentally a mathematical concept (a differentiable approximation to stack/queue/deque operations). The §101 defense rests on (1) the hardware tie in Claim 1's system format ("augmented neural network system" as a physical computer system), (2) the CRM format in Claim 20 which anchors to a tangible storage medium, and (3) the specific mathematical formula-based size vector update operations in Claims 4, 9, and 15 which may constitute a "particular machine" or "specific implementation." The method claim (Claim 12) is the weakest §101 position as it recites only abstract computational steps without an explicit hardware anchor in the independent claim body.

Explore in Eureka ↗

⚠️

Dependent Claim Fallback Quality

The dependent claim set has a structural redundancy problem: Claims 2–11 (dependent on Claim 1), Claims 13–19 (dependent on Claim 12), and the structure of Claim 20 all cover nearly identical technical limitations in a tripartite parallel structure, meaning the effective fallback depth is only about 4 meaningful technical refinements rather than 17 distinct positions. Claims 7–11 (deque embodiment) add genuine value as distinct technical limitations not in Claims 2–6 (stack embodiment). However, Claims 2 and 13 are essentially identical limitations expressed for system vs. method formats — a practitioner facing invalidation of Claim 1 would find Claim 12 provides no additional technical differentiation.

Explore in Eureka ↗

⚠️

Abstract Quality

An examiner reading only the abstract would correctly identify that the patent concerns augmenting neural networks with external memory, but would likely fail to identify the specific novel contribution — the size vector mechanism governing differentiable read/write weights that enables the memory to function as a continuous analogue of stack/queue/deque data structures. The abstract describes the operational steps at a high level ("determining memory state parameters," "updating the current state") without mentioning the size vector, the matrix memory structure, or the differentiable data structure analogy that is the true inventive contribution differentiating this from prior RNN-with-memory architectures.

Explore in Eureka ↗

✅

Figure Support Quality

The five figures provide adequate but not comprehensive coverage of the claimed embodiments. FIG. 1 supports the system architecture of Claim 1 (RNN 102, Memory Interface Subsystem 106, External Memory 104). FIGS. 3, 4, and 5 directly support the three dependent claim chains covering stack (Claims 3–6), queue (Claims 13–17 stack variant), and deque (Claims 7–11 and 18–19) memory types. The key gap is that no figure illustrates the mathematical size vector update operation at the formula level — Claims 4 and 9 recite specific size vector update formulas that have only textual specification support (columns 5–10) without a dedicated figure showing the mathematical relationships.

Explore in Eureka ↗

Analysis powered by PatSnap Eureka. Patent text and figures publicly available from USPTO. Draft a Similar Patent

Scorecard

Strategic Intent Scorecard

Multi-dimensional assessment of this application's patent strategy quality, based on claim structure, specification depth, and prosecution positioning.

Claim Breadth

3.5

Prosecution Defensibility

3.2

Spec–Claim Consistency

Dependent Claim Coverage

2.8

Claim Type Diversity

4.5

Figure Support Quality

3.5

Key observation: The strongest dimension is Claim Type Diversity (4.5/5.0) — the tripartite filing across Claims 1 (system), 12 (method), and 20 (CRM) provides enforcement coverage against hardware implementers, method practitioners, and software distributors simultaneously, which is exceptional for a continuation filing of this vintage. The weakest dimension is Dependent Claim Coverage (2.8/5.0) — the 17 dependent claims effectively deliver only 5–6 distinct technical positions because Claims 2–11 and 13–19 mirror each other's limitations across the system and method independent claims rather than adding independently novel technical refinements, meaning invalidation of a key dependent claim limitation would simultaneously eliminate its parallel counterpart across all three independent claim chains. Practitioners should note that a continuation application adding deque-specific independent claims could substantially improve the prosecution-defensible claim count.

See how your own draft compares — Open Eureka IP Drafting →

Critical Gaps

3 Critical Gaps in This Claim Set

A senior-attorney lens on the three highest-priority structural weaknesses — what each exposes in prosecution and litigation, and what a stronger filing would have done differently.

GAP 01 · HIGHEST IMPACT

No independent claim specific to differentiable deque memory architecture

The double-ended queue (deque) embodiment — which represents the most technically novel and complex memory architecture disclosed — is only protected through dependent Claims 7–11 (off Claim 1) and Claims 18–19 (off Claim 12), not through any independent claim. This creates a critical vulnerability: if Claim 1 or Claim 12 is successfully challenged on prior art grounds (e.g., using the Graves Neural Turing Machine reference cited on the face of the patent), the deque-specific limitations in Claims 7–11 fall entirely, leaving the deque innovation unprotected as an independent technology. A stronger filing would have included a fourth independent claim directed specifically to the continuous double-ended queue memory architecture with its top/bottom pop-push mechanism, given that this architecture is separately described in FIG. 5 and has distinct technical characteristics not shared with the stack and queue embodiments.

GAP 02 · HIGH IMPACT

Size vector update formula unclaimed at mathematical specificity level

The specification discloses mathematically precise formulas for the size vector update operations (columns 5–10, detailed equations for S_t[i] = max(0, S_{t-1}[i] - max(0, u - sum terms))) that define the differentiable analogue of pop operations, but no claim recites these formulas at a level of mathematical specificity that would capture the exact algorithmic implementation. Claims 4 and 15 recite only that "updating the entries of the size vector comprises updating the entries of the size vector using the pop value" — a functional description so broad that a competitor implementing a different differentiable deletion mechanism (e.g., using softmax weights instead of the described max-based formula) could design around the claims without escaping the broader functional language. A stronger filing would have included at least one dependent claim reciting the specific mathematical relationship used to update the size vector, creating a defensible fallback position if the broader functional claims are narrowed during post-grant proceedings.

GAP 03 · HIGH IMPACT

Training process for memory-augmented RNN parameters unclaimed

Unlock to read the full analysis.

🔒

3 Critical Gaps in This Claim Set

See the full attorney-level analysis of what this application leaves unprotected — and how to draft it more defensively for your own filings.

No independent deque-specific claim Size vector formula not claimed precisely Training/backpropagation method unclaimed

Unlock Full Analysis — Free

Frequently asked questions

US 12,099,928 B2 — key questions answered

What does US 12,099,928 B2 protect?+

Who owns patent US 12,099,928 B2?+

What are the independent claims of US 12,099,928 B2?+

What technology does this patent cover?+

What IPC classifications does this patent use?+

Still have questions? PatSnap Eureka can answer them from patent data instantly. Search in Eureka

Disclaimer: This analysis is generated by PatSnap Eureka AI based on publicly available patent data from the USPTO. It does not constitute legal advice and should not be relied upon as such. Patent data may be subject to change as prosecution progresses. Scores and assessments reflect automated analysis and may not capture all relevant legal or technical nuances. Always consult a qualified patent attorney for formal legal opinions on patentability, freedom to operate, or infringement.

Ask anything about this patent.
PatSnap Eureka searches patents and data to answer instantly.

Link copied to clipboard

AI AGENTS

AI APPLICATIONS

OTHERS

INDUSTRIES

USE CASES

EXPLORE

ENGAGE

SUPPORT & SERVICES

Your Agentic AI Partner
for Smarter Innovation

Great, Please verify your email.

Patent Drafting Analysis of DeepMind Technologies’ Augmented Recurrent Neural Network with External Memory | US 12,099,928 B2

Patent Drafting Analysis of DeepMind Technologies' Augmented Recurrent Neural Network with External Memory | US 12,099,928 B2