Datastore - Inconsistent behavior of transactions between Datastore and Firestore in Datastore Mode

[sai-pullabhotla]

I'm seeing inconsistency in transactions behavior between Cloud Datastore and Cloud Firestore running in Datastore mode. The sample code below runs fine when working with Cloud Datastore, but fails when working with Cloud Firestore running in Datastore mode.

import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.FullEntity;
import com.google.cloud.datastore.IncompleteKey;
import com.google.cloud.datastore.Transaction;

public class TransactionTest {

  public static void main(String[] args) {
    //change it to a project ID that has plain old Datastore and then try again 
    //with a different project that has Firestore running in Datastore mode. 
    final String projectId = "xxx-xxx";
    Datastore datastore = DatastoreOptions.newBuilder().setProjectId(projectId).build()
        .getService();
    IncompleteKey incompleteKey = datastore.newKeyFactory().setKind("TransactionTest").newKey();
    FullEntity<IncompleteKey> fullEntity = FullEntity.newBuilder().setKey(incompleteKey)
        .set("name", "Bob").build();
    com.google.cloud.datastore.Entity entity = datastore.add(fullEntity);

    Transaction t1 = datastore.newTransaction();
    try {
      com.google.cloud.datastore.Entity entity2 = t1.get(entity.getKey());
      update(entity, datastore);
      t1.commit();
      System.out.println("t1 is committed successfully");
    } finally {
      if (t1.isActive()) {
        t1.rollback();
      }
    }

  }

  private static void update(com.google.cloud.datastore.Entity entity, Datastore datastore) {
    Transaction t2 = datastore.newTransaction();
    try {
      t2.update(entity);
      t2.commit();
      System.out.println("t2 is committed successfully");
    } finally {
      if (t2.isActive()) {
        t2.rollback();
      }
    }
  }

}

Running this code against good old Datastore completes normally and I get the below printed to the console:

t2 is committed successfully
t1 is committed successfully

However, running this against a different project_id, which has Firestore running in Datastore mode, produces the below exception, after running for about a minute:

Exception in thread "main" com.google.cloud.datastore.DatastoreException: The referenced transaction has expired or is no longer valid.
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:114)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.rollback(HttpDatastoreRpc.java:173)
	at com.google.cloud.datastore.DatastoreImpl$6.call(DatastoreImpl.java:530)
	at com.google.cloud.datastore.DatastoreImpl$6.call(DatastoreImpl.java:527)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
	at com.google.cloud.datastore.DatastoreImpl.rollback(DatastoreImpl.java:527)
	at com.google.cloud.datastore.DatastoreImpl.rollbackTransaction(DatastoreImpl.java:522)
	at com.google.cloud.datastore.TransactionImpl.rollback(TransactionImpl.java:120)
	at com.jmethods.catatumbo.TransactionTest.main(TransactionTest.java:30)
Caused by: com.google.datastore.v1.client.DatastoreException: The referenced transaction has expired or is no longer valid., code=INVALID_ARGUMENT
	at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:226)
	at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:275)
	at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:186)
	at com.google.datastore.v1.client.Datastore.rollback(Datastore.java:111)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.rollback(HttpDatastoreRpc.java:171)
	... 9 more

Any idea why this difference? Which one is behaving correctly, Datastore or Firestore in Datastore mode?

[athakor]

@sai-pullabhotla

As per the google documentation Firestore in datastore mode has all the existing functionality of google cloud datastore with extra features.Currently you are getting DatastoreException "The referenced transaction has expired or is no longer valid." just because of the entity that you are using for transactions.

For the transaction t2 you change the nature of entity t2.update(entity) and then commit transaction t2. Transaction t1 also refer same entity t1.get(entity.getKey()) and while committing t1 you will get Datastore exception.Could you please check method level comment of get(Key key) method.

• Reference : https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-clients/google-cloud-datastore/src/main/java/com/google/cloud/datastore/Transaction.java#L82

For the transaction T1 could you please modified your code something like below because it's working fine in both project cloud datastore and firestore in datastore mode.

try {
      update(entity, datastore);
      com.google.cloud.datastore.Entity entity2 = t1.get(entity.getKey());
      t1.commit();
      System.out.println("t1 is committed successfully");
    } finally {
      if (t1.isActive()) {
        t1.rollback();
      }
    }

[athakor]

closing this issue for now due to staleness and if above reply didn't help feel free to reopen it.