alonisser 10 hours ago

What I’ve learned about spark streaming checkpointing, how it’s implemented, and how the implementation details of offsets might lead to surprising behavior.