Speeding up schema generation during tests

[Ali Akhtar]

Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[DuyHai Doan]

There is a lot of pre-flight checks when starting the cassandra server and
they took time.

For integration testing, I have developped a modified CassandraDeamon here
that remove pretty most of those checks:

https://github.com/doanduyhai/Achilles/blob/master/achilles-embedded/src/main/java/info/archinnov/achilles/embedded/AchillesCassandraDaemon.java

The problem is that I felt into weird scenarios where creating a keyspace
wasn't created in timely manner so I just stop using this impl for the
moment, just look at it and do whatever you want.

Another idea for testing is to disable durable write to speed up mutation
(CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[Ali Akhtar]

Thanks. I've disabled durable writes but this is still pretty slow (about
10 seconds).

What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server and
they took time.
For integration testing, I have developped a modified CassandraDeamon here
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/
AchillesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a keyspace
wasn't created in timely manner so I just stop using this impl for the
moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up mutation
(CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[DuyHai Doan]

As I said, when I bootstrap the server and create some keyspace, sometimes
the schema is not fully initialized and when the test code tried to insert
data, it fails.

I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow (about
10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server
and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a keyspace
wasn't created in timely manner so I just stop using this impl for the
moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up mutation
(CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[horschi]

Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace, sometimes
the schema is not fully initialized and when the test code tried to insert
data, it fails.
I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow (about
10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server
and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[Ali Akhtar]

Horschi, you are the hero gotham deserves.

Test time reduced from 10 seconds to 800 ms

Post by horschi
Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace,
sometimes the schema is not fully initialized and when the test code tried
to insert data, it fails.
I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow
(about 10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server
and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[DuyHai Doan]

Ohh didn't know such system property exist, nice idea!

Post by horschi
Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace,
sometimes the schema is not fully initialized and when the test code tried
to insert data, it fails.
I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow
(about 10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server
and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[DuyHai Doan]

Ok I have added -Dcassandra.unsafesystem=true and my tests are broken.

The reason is that I create some schemas before executing tests.

When unable unsafesystem, Cassandra does not block for schema flush so you
man run into race conditions where the test start using the created schema
but it has not been fully flushed yet to disk:

See C* source code here:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L278-L282

static void flush()
{
if (!DatabaseDescriptor.isUnsafeSystem())
ALL.forEach(table ->
FBUtilities.waitOnFuture(getSchemaCFS(table).forceFlush()));
}

I don't know how it worked out for you but it didn't for me...

Post by DuyHai Doan
Ohh didn't know such system property exist, nice idea!

Post by horschi
Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace,
sometimes the schema is not fully initialized and when the test code tried
to insert data, it fails.
I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow
(about 10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra server
and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[Ali Akhtar]

I'm using https://github.com/jsevellec/cassandra-unit and haven't come
across any race issues or problems. Cassandra-unit takes care of creating
the schema before it runs the tests.

Post by DuyHai Doan
Ok I have added -Dcassandra.unsafesystem=true and my tests are broken.
The reason is that I create some schemas before executing tests.
When unable unsafesystem, Cassandra does not block for schema flush so you
man run into race conditions where the test start using the created schema
See C* source code here: https://github.com/apache/cassandra/blob/trunk/
src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L278-L282
static void flush()
{
if (!DatabaseDescriptor.isUnsafeSystem())
ALL.forEach(table -> FBUtilities.waitOnFuture(
getSchemaCFS(table).forceFlush()));
}
I don't know how it worked out for you but it didn't for me...

Post by DuyHai Doan
Ohh didn't know such system property exist, nice idea!

Post by horschi
Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace,
sometimes the schema is not fully initialized and when the test code tried
to insert data, it fails.
I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow
(about 10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra
server and they took time.
For integration testing, I have developped a modified CassandraDeamon
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.

[horschi]

You have to manually do "nodetool flush && nodetool flush system" before
shutdown, otherwise Cassandra might break. With that it is working nicely.

Post by Ali Akhtar
I'm using https://github.com/jsevellec/cassandra-unit and haven't come
across any race issues or problems. Cassandra-unit takes care of creating
the schema before it runs the tests.

Post by DuyHai Doan
Ok I have added -Dcassandra.unsafesystem=true and my tests are broken.
The reason is that I create some schemas before executing tests.
When unable unsafesystem, Cassandra does not block for schema flush so
you man run into race conditions where the test start using the created
See C* source code here: https://github.com/apach
e/cassandra/blob/trunk/src/java/org/apache/cassandra/sche
ma/SchemaKeyspace.java#L278-L282
static void flush()
{
if (!DatabaseDescriptor.isUnsafeSystem())
ALL.forEach(table -> FBUtilities.waitOnFuture(getSc
hemaCFS(table).forceFlush()));
}
I don't know how it worked out for you but it didn't for me...

Post by DuyHai Doan
Ohh didn't know such system property exist, nice idea!

Post by horschi
Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?

Post by DuyHai Doan
As I said, when I bootstrap the server and create some keyspace,
sometimes the schema is not fully initialized and when the test code tried
to insert data, it fails.
I did not have time to dig into the source code to find the root
cause, maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

Post by Ali Akhtar
Thanks. I've disabled durable writes but this is still pretty slow
(about 10 seconds).
What issues did you run into with your impl?

Post by DuyHai Doan
There is a lot of pre-flight checks when starting the cassandra
server and they took time.
For integration testing, I have developped a modified
https://github.com/doanduyhai/Achilles/blob/master/achilles-
embedded/src/main/java/info/archinnov/achilles/embedded/Achi
llesCassandraDaemon.java
The problem is that I felt into weird scenarios where creating a
keyspace wasn't created in timely manner so I just stop using this impl for
the moment, just look at it and do whatever you want.
Another idea for testing is to disable durable write to speed up
mutation (CREATE KEYSPACE ... WITH durable_write=false)

Post by Ali Akhtar
Is there a way to speed up the creation of keyspace + tables during
integration tests? I am using an RF of 1, with SimpleStrategy, but it still
takes upto 10-15 seconds.