260 lines
6.5 KiB
Markdown
260 lines
6.5 KiB
Markdown
|
||
# 1) Data model & basics
|
||
|
||
* A **queue** is a List at key `queue:<name>`.
|
||
* Common patterns:
|
||
|
||
* **Producer**: `LPUSH queue item` (or `RPUSH`)
|
||
* **Consumer (non-blocking)**: `RPOP queue` (or `LPOP`)
|
||
* **Consumer (blocking)**: `BRPOP queue timeout` (or `BLPOP`)
|
||
* If a key doesn’t exist, it’s treated as an **empty list**; push **creates** the list; when the **last element is popped, the key is deleted**. ([Redis][1])
|
||
|
||
---
|
||
|
||
# 2) Commands to implement (queues via Lists)
|
||
|
||
## LPUSH / RPUSH
|
||
|
||
Prepend/append one or more elements. Create the list if it doesn’t exist.
|
||
**Return**: Integer = new length of the list.
|
||
|
||
**Syntax**
|
||
|
||
```
|
||
LPUSH key element [element ...]
|
||
RPUSH key element [element ...]
|
||
```
|
||
|
||
**RESP (example)**
|
||
|
||
```
|
||
*3\r\n$5\r\nLPUSH\r\n$5\r\nqueue\r\n$5\r\njob-1\r\n
|
||
:1\r\n
|
||
```
|
||
|
||
Refs: semantics & multi-arg ordering. ([Redis][1])
|
||
|
||
### LPUSHX / RPUSHX (optional but useful)
|
||
|
||
Like LPUSH/RPUSH, **but only if the list exists**.
|
||
**Return**: Integer = new length (0 if key didn’t exist).
|
||
|
||
```
|
||
LPUSHX key element [element ...]
|
||
RPUSHX key element [element ...]
|
||
```
|
||
|
||
Refs: command index. ([Redis][2])
|
||
|
||
---
|
||
|
||
## LPOP / RPOP
|
||
|
||
Remove & return one (default) or **up to COUNT** elements since Redis 6.2.
|
||
If the list is empty or missing, **Null** is returned (Null Bulk or Null Array if COUNT>1).
|
||
**Return**:
|
||
|
||
* No COUNT: Bulk String or Null Bulk.
|
||
* With COUNT: Array of Bulk Strings (possibly empty) or Null Array if key missing.
|
||
|
||
**Syntax**
|
||
|
||
```
|
||
LPOP key [count]
|
||
RPOP key [count]
|
||
```
|
||
|
||
**RESP (no COUNT)**
|
||
|
||
```
|
||
*2\r\n$4\r\nRPOP\r\n$5\r\nqueue\r\n
|
||
$5\r\njob-1\r\n # or $-1\r\n if empty
|
||
```
|
||
|
||
**RESP (COUNT=2)**
|
||
|
||
```
|
||
*3\r\n$4\r\nLPOP\r\n$5\r\nqueue\r\n$1\r\n2\r\n
|
||
*2\r\n$5\r\njob-2\r\n$5\r\njob-3\r\n # or *-1\r\n if key missing
|
||
```
|
||
|
||
Refs: LPOP w/ COUNT; general pop semantics. ([Redis][3])
|
||
|
||
---
|
||
|
||
## BLPOP / BRPOP (blocking consumers)
|
||
|
||
Block until an element is available in any of the given lists or until `timeout` (seconds, **double**, `0` = forever).
|
||
**Return** on success: **Array \[key, element]**.
|
||
**Return** on timeout: **Null Array**.
|
||
|
||
**Syntax**
|
||
|
||
```
|
||
BLPOP key [key ...] timeout
|
||
BRPOP key [key ...] timeout
|
||
```
|
||
|
||
**RESP**
|
||
|
||
```
|
||
*3\r\n$5\r\nBRPOP\r\n$5\r\nqueue\r\n$1\r\n0\r\n # block forever
|
||
|
||
# Success reply
|
||
*2\r\n$5\r\nqueue\r\n$5\r\njob-4\r\n
|
||
|
||
# Timeout reply
|
||
*-1\r\n
|
||
```
|
||
|
||
**Implementation notes**
|
||
|
||
* If any listed key is non-empty at call time, reply **immediately** from the first non-empty key **by the command’s key order**.
|
||
* Otherwise, put the client into a **blocked state** (register per-key waiters). On any `LPUSH/RPUSH` to those keys, **wake the earliest waiter** and serve it atomically.
|
||
* If timeout expires, return **Null Array** and clear the blocked state.
|
||
Refs: timeout semantics and return shape. ([Redis][4])
|
||
|
||
---
|
||
|
||
## LMOVE / BLMOVE (atomic move; replaces RPOPLPUSH/BRPOPLPUSH)
|
||
|
||
Atomically **pop from one side** of `source` and **push to one side** of `destination`.
|
||
|
||
* Use for **reliable queues** (move to a *processing* list).
|
||
* `BLMOVE` blocks like `BLPOP` when `source` is empty.
|
||
|
||
**Syntax**
|
||
|
||
```
|
||
LMOVE source destination LEFT|RIGHT LEFT|RIGHT
|
||
BLMOVE source destination LEFT|RIGHT LEFT|RIGHT timeout
|
||
```
|
||
|
||
**Return**: Bulk String element moved, or Null if `source` empty (LMOVE); `BLMOVE` blocks/Null on timeout.
|
||
|
||
**RESP (LMOVE RIGHT->LEFT)**
|
||
|
||
```
|
||
*5\r\n$5\r\nLMOVE\r\n$6\r\nsource\r\n$3\r\ndst\r\n$5\r\nRIGHT\r\n$4\r\nLEFT\r\n
|
||
$5\r\njob-5\r\n
|
||
```
|
||
|
||
**Notes**
|
||
|
||
* Prefer `LMOVE/BLMOVE` over deprecated `RPOPLPUSH/BRPOPLPUSH`.
|
||
* Pattern: consumer `LMOVE queue processing RIGHT LEFT` → work → `LREM processing 1 <elem>` to ACK; a reaper can requeue stale items.
|
||
Refs: LMOVE/BLMOVE behavior and reliable-queue pattern; deprecation of RPOPLPUSH. ([Redis][5])
|
||
|
||
*(Compat: you can still implement `RPOPLPUSH source dest` and `BRPOPLPUSH source dest timeout`, but mark them deprecated and map to LMOVE/BLMOVE.)* ([Redis][6])
|
||
|
||
---
|
||
|
||
## LLEN (length)
|
||
|
||
Useful for metrics/backpressure.
|
||
|
||
```
|
||
LLEN key
|
||
```
|
||
|
||
**RESP**
|
||
|
||
```
|
||
*2\r\n$4\r\nLLEN\r\n$5\r\nqueue\r\n
|
||
:3\r\n
|
||
```
|
||
|
||
Refs: list overview mentioning LLEN. ([Redis][7])
|
||
|
||
---
|
||
|
||
## LREM (ack for “reliable” processing)
|
||
|
||
Remove occurrences of `element` from the list (head→tail scan).
|
||
Use `count=1` to ACK a single processed item from `processing`.
|
||
|
||
```
|
||
LREM key count element
|
||
```
|
||
|
||
**RESP**
|
||
|
||
```
|
||
*4\r\n$4\r\nLREM\r\n$9\r\nprocessing\r\n$1\r\n1\r\n$5\r\njob-5\r\n
|
||
:1\r\n
|
||
```
|
||
|
||
Refs: reliable pattern mentions LREM to ACK. ([Redis][5])
|
||
|
||
---
|
||
|
||
## LTRIM (bounded queues / retention)
|
||
|
||
Keep only `[start, stop]` range; everything else is dropped.
|
||
Use to cap queue length after pushes.
|
||
|
||
```
|
||
LTRIM key start stop
|
||
```
|
||
|
||
**RESP**
|
||
|
||
```
|
||
*4\r\n$5\r\nLTRIM\r\n$5\r\nqueue\r\n$2\r\n0\r\n$3\r\n999\r\n
|
||
+OK\r\n
|
||
```
|
||
|
||
Refs: list overview includes LTRIM for retention. ([Redis][7])
|
||
|
||
---
|
||
|
||
## LRANGE / LINDEX (debugging / peeking)
|
||
|
||
* `LRANGE key start stop` → Array of elements (non-destructive).
|
||
* `LINDEX key index` → one element or Null.
|
||
|
||
These aren’t required for queue semantics, but handy. ([Redis][7])
|
||
|
||
---
|
||
|
||
# 3) Errors & types
|
||
|
||
* Wrong type: `-WRONGTYPE Operation against a key holding the wrong kind of value\r\n`
|
||
* Non-existing key:
|
||
|
||
* Push: creates the list (returns new length).
|
||
* Pop (non-blocking): returns **Null**.
|
||
* Blocking pop: **Null Array** on timeout. ([Redis][1])
|
||
|
||
---
|
||
|
||
# 4) Blocking engine (implementation sketch)
|
||
|
||
1. **Call time**: scan keys in user order. If a non-empty list is found, pop & reply immediately.
|
||
2. **Otherwise**: register the client as **blocked** on those keys with `deadline = now + timeout` (or infinite).
|
||
3. **On push to any key**: if waiters exist, **wake one** (FIFO) and serve its pop **atomically** with the push result.
|
||
4. **On timer**: for each blocked client whose deadline passed, reply `Null Array` and clear state.
|
||
5. **Connection close**: remove from any wait queues.
|
||
|
||
Refs for timeout/block semantics. ([Redis][4])
|
||
|
||
---
|
||
|
||
# 5) Reliable queue pattern (recommended)
|
||
|
||
* **Consume**: `LMOVE queue processing RIGHT LEFT` (or `BLMOVE ... 0`).
|
||
* **Process** the job.
|
||
* **ACK**: `LREM processing 1 <job>` when done.
|
||
* **Reaper**: auxiliary task that detects stale jobs (e.g., track job IDs + timestamps in a ZSET) and requeues them. (Lists don’t include timestamps; pairing with a ZSET is standard practice.)
|
||
Refs: LMOVE doc’s pattern. ([Redis][5])
|
||
|
||
---
|
||
|
||
# 6) Minimal test matrix
|
||
|
||
* Push/pop happy path (both ends), with/without COUNT.
|
||
* Blocking pop: immediate availability, block + timeout, wake on push, multiple keys order, FIFO across multiple waiters.
|
||
* LMOVE/BLMOVE: RIGHT→LEFT pipeline, block + wake, cross-list atomicity, ACK via LREM.
|
||
* Type errors and key deletion on last pop.
|
||
|