Gsi Technology, Inc. · NASDAQ:GSIT
Gsi Technology — REWORK (October 2021)
19 slides from Gsi Technology's conference presentation, October 2021
Slide 1: In-Place Deep Learning
In-Place Deep Learning Dr. Avidan Akerib VP Associative Computing BU 1
Slide 2: CPU-GPGPU Solution
CPU-GPGPU Solution Question General CPU Purpose Very Wide Bus DRAM GPU Answer • Creates I/O bottleneck between the processor and memory • Limits performance • Increases power consumption • Does not scale with the search, sort, and rank requirements of applications like recommender systems and data m
Slide 3: Brain Computes in Place
Brain Computes in Place COMMITMENT INNOVATION SPEED
Slide 4: GSI's Solution: APU—Associative Processing Unit
GSI’s Solution APU—Associative Processing Unit Associative Memory Question Simple Simple & Narrow CPU Bus APU Answer • Computes in-place directly in the memory array—removes the I/O bottleneck • Significantly increases performance • Reduces power 4
Slide 5: Computing in the Bit Lines
Computing in the Bit Lines a0 a1 a2 a3 a4 a5 a6 a7 A b0 b1 b2 b3 b4 b5 b6 b7 B C=f(A,B) Each bit line becomes a processor and storage Millions of bit lines = millions of processors 5
Slide 6: Neighborhood Computing
Neighborhood Computing a0 a1 a2 a3 a4 a5 a6 a7 A b0 b1 b2 b2 b0 b1 b3 b4 b5 b5 b3 b4 b6 b6 b7 b7 SL(B,1) C=f(A,SL(B,1)) Parallel shift of bit lines @ 1 cycle sections Enables neighborhood operations such as convolutions 6
Slide 7: Search & Count
Search & Count 5 20 12 54 -17 20 20 1 Search 20 Count = 3 1 1 1 • Search (binary or ternary) all bit lines in 1 cycle • 128 M bit lines => 128 Peta search/sec • Key Applications for search and count: • Recommender systems • K-Nearest Neighbors (using cosine similarity search) • Random forest • Image
Slide 8: APU Card
APU Card APU (2M processors) APU APU APU System Controller Chip APU APU APU APU PCIe • PCI card size holds up to 8 APUs • 125 Tera OPS @ 25 Watts (average of search, add, sub, FP mul, etc) • 5 Tera OPS/Watt 8
Slide 9: APU Server
APU Server PCIe 8 APU Cards : 1 Peta OPS/sec @ 200 W 9
Slide 10: K-Nearest Neighbors (k-NN)
K-Nearest Neighbors (k-NN) Simple example: N = 36, 3 Groups, 2 dimensions (D = 2 ) for X and Y K=4 Group Green selected as the majority For actual applications: N = Billions, D = Tens, K = Tens of thousands 10
Slide 11: k-NN use case in an APU
k-NN use case in an APU Item 1 Features for Item N Item Features for Item 1 Features for Item 2 N Item Item features and label 2 storage Item 3 Compute cosine distances for all N in parallel (≤ 10 µ sec, assuming D = 50 features) Q Distribute data – 2 nsec ( to all) Computing Area K Mins at O(1) com
Slide 12: Large Database Example Using APU Servers
Large Database Example Using APU Servers • Number of items: Billions • Features per item: tens to hundreds • Latency: ≤ 1 msec • Throughput: Scales to 1M similarity searches/sec • k-NN: Top 100,000 nearest neighbors 12
Slide 13: Example k-NN for Recognition
Example k-NN for Recognition Recognition/ Signal Features Classification Feature K-NN Extraction Classifier Image Result of CNN after Recognize/classify convolution layers image Speech detection, Speech MFCC noise cancelation… IG, CHI, MI, Classify Documents NGL, GSS, RS, documents, spam MSF DF, det
Slide 14: Image Recognition using k-NN
Image Recognition using k-NN Features of image N Features of image 1 Features of image 2 Image features computed at the final stage of convolution layer + preprocesses related to all images in data base Convolution Layer 𝑝𝑝 𝐷𝐷𝑝𝑝 �𝑄𝑄 ∑𝑛𝑛 𝑖𝑖=0 𝐷𝐷𝑖𝑖 𝑄𝑄𝑖𝑖 Compute cosine distances…
Slide 15: K-MINS: O(1) Algorithm
K-MINS: O(1) Algorithm MSB LSB KMINS(int K, vector C){ C0 M := 1, V := 0; C1 FOR b = msb to b = lsb: D := not(C[b]); C2 N := M & D; cnt = COUNT(N|V) IF cnt > K: N M := N; ELIF cnt < K: V := N|V; ELSE: // cnt == K V := N|V; EXIT; ENDIF ENDFOR } 15
Slide 16: K-MINS: The Algorithm
K-MINS: The Algorithm V C 0 110101 KMINS(int K, vector C){ 0 010100 M := 1, V := 0; 1 000101 FOR b = msb to b = lsb: D := not(C[b]); 0 000111 N := M & D; 1 000001 cnt = COUNT(N|V) 0 111011 IF cnt > K: 1 000101 M := N; 0 010110 ELIF cnt < K: 0 001100 V := N|V; 0 011100 ELSE: // cnt == K 0 111100 V :=
Slide 17: Matrix Multiplication: Dense Vector by Sparse Matrix
Matrix Multiplication Dense (1XN) Vector by Sparse NxM Input Vector APU Representations and Computing 4 -2 3 -1 3 5 9 17 Sparse Matrix Column 1 2 2 4 0 0 0 0 Row 2 3 4 4 3 0 0 0 Search all columns for row = 2 :Distribute -2 : 2 Cy 0 5 0 0 -2 3 -1 -1 Search all columns for row = 3 :Distribute -3 : 2
Slide 18: Sparse Matrix Multiplication Performance Analysis
Sparse Matrix Multiplication Performance Analysis • G3 circuit matrix • 1.5M X 1.5M sparse matrix • Roughly 8M nonzero elements • 20–100 GFLOPS with GPGPU solution • APU solution provides 64 TFLOPS using the same amount of power as the GPGPU solution above • > 500x improvement with APU solution 18
Slide 19: Summary
Summary • Access data by content not by address • In-place computation of randomly distributed data using millions of parallel processors • >500x performance-over-power improvement for recommender systems and data mining • >500x performance improvement for sparse matrix multiplication compared to GP