Discussion:
how to immediately delete tombstones
onmstester onmstester
2018-05-31 15:07:38 UTC
Permalink
Hi,

I've deleted 50% of my data row by row now disk usage of cassandra data is more than 80%.

The gc_grace of table was default (10 days), now i set that to 0, although many compactions finished but no space reclaimed so far.

How could i force deletion of tombstones in sstables and reclaim the disk used by deleted rows?

I'm using cassandra on a single node.
Sent using Zoho Mail
Nicolas Guyomar
2018-05-31 15:19:23 UTC
Permalink
Hi,

You need to manually force compaction if you do not care ending up with one
big sstable (nodetool compact)
Post by onmstester onmstester
Hi,
I've deleted 50% of my data row by row now disk usage of cassandra data is more than 80%.
The gc_grace of table was default (10 days), now i set that to 0, although
many compactions finished but no space reclaimed so far.
How could i force deletion of tombstones in sstables and reclaim the disk
used by deleted rows?
I'm using cassandra on a single node.
Sent using Zoho Mail <https://www.zoho.com/mail/>
Alain RODRIGUEZ
2018-05-31 18:02:32 UTC
Permalink
Hello,

It's a very common but somewhat complex topic. We wrote about it 2 years
ago and I really think this post might have answers you are looking for:
http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html

Something that you could try (if you do care ending up with one big sstable
:)) is to enable 'unchecked_tombstone_compaction'. This removes a pre-check
supposed to only trigger tombstone compactions when it is worth it (ie
there is no sstable overlaps) but that over time proved to be inefficient
from a tombstone eviction perspective.

If data can actually be deleted (no overlaps, gc_grace_seconds lowered or
reached, ...) then changing option 'unchecked_tombstone_compaction' to
'true' might do a lot of good in terms of disk space. Be aware that a bunch
of compactions might be triggered and that disk space will start by
increasing before (possibly) reducing after the compactions.

The gc_grace of table was default (10 days), now i set that to 0, although
Post by onmstester onmstester
many compactions finished but no space reclaimed so far.
Be aware that if some deletes did not reach all the replicas, the data will
eventually come back as lowering the gc_grace_seconds, you don't allow
repairs to process the data before tombstones are actually evicted.

Also, by setting the gc_grace_seconds to 0, you also disabled the hints
altogether. gc_grace_seconds should always be equivalent to
'max_hint_windows_in_ms'. My colleague Radovan wrote a post with
more information on this:
http://thelastpickle.com/blog/2018/03/21/hinted-handoff-gc-grace-demystified.html

Good luck with your tombstones, again, those are a bit tricky to handle
sometimes ;-)

C*heers,
-----------------------
Alain Rodriguez - @arodream - ***@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com
Post by onmstester onmstester
Hi,
You need to manually force compaction if you do not care ending up with
one big sstable (nodetool compact)
Post by onmstester onmstester
Hi,
I've deleted 50% of my data row by row now disk usage of cassandra data is more than 80%.
The gc_grace of table was default (10 days), now i set that to 0,
although many compactions finished but no space reclaimed so far.
How could i force deletion of tombstones in sstables and reclaim the disk
used by deleted rows?
I'm using cassandra on a single node.
Sent using Zoho Mail <https://www.zoho.com/mail/>
onmstester onmstester
2018-06-02 04:04:22 UTC
Permalink
Thanks for your replies

But my current situation is that i do not have enough free disk for my biggest sstable, so i could not run major compaction or nodetool garbagecollect


Sent using Zoho Mail






---- On Thu, 31 May 2018 22:32:32 +0430 Alain RODRIGUEZ &lt;***@gmail.com&gt; wrote ----
Nitan Kainth
2018-06-02 07:29:17 UTC
Permalink
You can compact selective sstables using jmx Call.

Sent from my iPhone
Post by onmstester onmstester
Thanks for your replies
But my current situation is that i do not have enough free disk for my biggest sstable, so i could not run major compaction or nodetool garbagecollect
Sent using Zoho Mail
Continue reading on narkive:
Loading...