Attempting to read from a non-existent database can create many distinct client IDs in the session metrics

[sebright2]

I noticed that calling Spanner.getDatabaseClient with a non-existent database and then trying to read from the database can cause new client IDs to appear in the OpenCensus session metrics. The client IDs for the non-existent database seem problematic because they can cause the size of the metrics to grow.

Here is a simplified reproduction:

google-cloud-spanner version: 1.54.0
opencensus-java version: 0.26.0

import com.google.cloud.spanner.DatabaseId;
import com.google.cloud.spanner.InstanceConfigId;
import com.google.cloud.spanner.InstanceId;
import com.google.cloud.spanner.InstanceInfo;
import com.google.cloud.spanner.Key;
import com.google.cloud.spanner.Spanner;
import com.google.cloud.spanner.SpannerOptions;
import io.opencensus.exporter.stats.stackdriver.StackdriverStatsExporter;
import java.util.Arrays;
import org.junit.Test;

public class ClientIdTest {
  @Test
  public void testClientId() throws Exception {
    StackdriverStatsExporter.createAndRegister();
    Spanner spanner = SpannerOptions.getDefaultInstance().getService();
    String projectId = spanner.getOptions().getProjectId();
    InstanceId instanceId = InstanceId.of(projectId, "my-instance");
    InstanceConfigId instanceConfigId = InstanceConfigId.of(projectId, "regional-us-central1");
    spanner
        .getInstanceAdminClient()
        .createInstance(
            InstanceInfo.newBuilder(instanceId)
                .setInstanceConfigId(instanceConfigId)
                .setDisplayName("my-instance")
                .setNodeCount(1)
                .build())
        .get();
    DatabaseId databaseId = DatabaseId.of(instanceId, "my-database");

    for (int i = 0; i < 100; i++) {
      try {
        spanner
            .getDatabaseClient(databaseId)
            .singleUse()
            .readRow("MyTable", Key.of(0), Arrays.asList("MyColumn"));
      } catch (Exception e) {
        // ignore
      }
    }
    Thread.sleep(300_000);
  }
}

The metrics exported to Stackdriver contain 100 client IDs. For example, here is in_use_sessions:

[sebright2]

I tried updating to master after #206 was merged, but I'm still getting a large number of client IDs. I also tried running the reproduction above, but this time the client IDs ranged from 1 to 50 instead of 1 to 100.

[olavloite]

@sebright
I'm not really able to reproduce this locally. I've just added an additional test case based on your reproduction code above, but I'm seeing the same client id each time.

The client ids in the range 1 to 50 especially baffles me. Is your test code literally equal to the reproduction code you mentioned above, or could it be that the test code is somehow running parts of it in parallel?

[olavloite]

@sebright Friendly ping. Is this still an issue?