fatal error org.bouncycastle.asn1.ASN1Exception after upgrade from version 1.60 to 1.71 or 1.77

[ivkina]

krb.txt

Please, see krb.txt uploaded to the case. The file contains standard Windows Domain Controller Kerberos ticket issued by Windows AD to a specific user account. It is a corp network, all patches and versions are up to date. The sample code below works just fine with version 1.60 (bcprov-jdk15on-1.60.jar). At some point after version 1.60, the implementation of ASN1InputStream.readObject() changed in a way that it no longer works. Verified that version 1.77 fails. Version 1.71 fails as well (bcprov-jdk18on-1.71).

The production use case is of course is more complex vs what the sample code below does. In production it parses out an then decyptes kerberos/spnego token to extract Windows PAC data (user authorization data aka user groups, roles etc).

package com;

import java.io.File;
import java.io.FileInputStream;
import java.security.Security;

import org.bouncycastle.asn1.ASN1InputStream;
import org.bouncycastle.asn1.ASN1Primitive;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

public class TestASN1 {

	public static void main(String[] args) {
		File f = new File("c:/temp/krb.txt");
		FileInputStream fis = null;
		ASN1InputStream ais = null;
		try {
			Security.addProvider(new BouncyCastleProvider());
			
			fis = new FileInputStream(f);
			ais = new ASN1InputStream(fis);
			ASN1Primitive asn1p = ais.readObject();
			System.out.println(asn1p);
		}
		catch (Exception e) {
			e.printStackTrace();
		}
		finally {
			try {
				if (ais != null) {
					ais.close();
				}
				if (fis != null) {
					fis.close();
				}
			}
			catch (Exception e) {
			}

		}
	}

}

The code throws an error on line ASN1Primitive asn1p = ais.readObject();. Stack trace:

org.bouncycastle.asn1.ASN1Exception: corrupted stream detected
	at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readVector(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readVector(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readTaggedObjectDL(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.buildObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source)
	at com.TestASN1.main(TestASN1.java:22)
Caused by: java.lang.IllegalArgumentException: BOOLEAN value should have 1 byte in it
	at org.bouncycastle.asn1.ASN1Boolean.createPrimitive(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.createPrimitiveDERObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.buildObject(Unknown Source)
	... 7 more

---

Edited for formatting by @cipherboy.

[dghgit]

We've been tightening up the ASN.1 parsing. The boolean at the start of the file is invalid - it's zero length! Are you seriously saying this is what Microsoft is producing? Even the openssl parser complains about it, and they're a lot more forgiving than we are.

[cipherboy]

@dghgit I don't think this is quite right. Note here:

https://datatracker.ietf.org/doc/html/rfc4121#section-4.1

The format of this message is:

         GSS-API DEFINITIONS ::=

         BEGIN

         MechType ::= OBJECT IDENTIFIER
         -- representing Kerberos V5 mechanism

         GSSAPI-Token ::=
         -- option indication (delegation, etc.) indicated within
         -- mechanism-specific token
         [APPLICATION 0] IMPLICIT SEQUENCE {
                 thisMech MechType,
                 innerToken ANY DEFINED BY thisMech
                    -- contents mechanism-specific
                    -- ASN.1 structure not required
                 }

   The innerToken field starts with a two-octet token-identifier
   (TOK_ID) expressed in big-endian order, followed by a Kerberos
   message.

          Token               TOK_ID Value in Hex
         -----------------------------------------
          KRB_AP_REQ            01 00
          KRB_AP_REP            02 00
          KRB_ERROR             03 00

What you read as a zero byte boolean, isn't; it is a two byte token signifying the content type of the remaining part of the ANY. 01 00 just happens to look unfortunately a lot like a zero-length ASN.1 boolean... :-)

This means a general purpose recursive, greedy ASN.1 Parser isn't the correct way to parse this; we've got to use something incremental to handle the GSSAPI specifics, or explicitly validate the outer packet. Given we know what the constructed prefix should be (60 82 09 18 06 09 2a 86 48 86 f7 12 01 02 02 01 00) and there's no suffix...

@ivkina I'd perhaps suggest reading and verifying that prefix manually, given it is unlikely to change for your purposes:

Code & output

Code:

import java.io.File;
import java.io.FileInputStream;
import java.security.Security;
import java.util.Arrays;

import org.bouncycastle.asn1.*;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

public class Main {

    public static void main(String[] args) {
        File f = new File("/home/ascheel/Downloads/krb.txt");

        FileInputStream fis = null;
        ASN1InputStream ais = null;
        try {
            Security.addProvider(new BouncyCastleProvider());
            fis = new FileInputStream(f);

            byte[] expected = new byte[]{0x60, (byte)0x82, 0x09, 0x18, 0x06,
                                         0x09, 0x2a, (byte)0x86, 0x48,
                                         (byte)0x86, (byte)0xf7, 0x12, 0x01,
                                         0x02, 0x02, 0x01, 0x00};
            byte[] actual = new byte[expected.length];
            if (fis.read(actual) != actual.length) {
                throw new RuntimeException("FIXME: input buffer too short");
            }
            if (!Arrays.equals(actual, expected)) {
                throw new RuntimeException("FIXME: input buffer has incorrect prefix");
            }

            // Now we can parse the inner KRB_AP_REQ directly.
            ais = new ASN1InputStream(fis);
            ASN1Primitive asn1p = ais.readObject();
            System.out.println(asn1p);
        }
        catch (Exception e) {
            e.printStackTrace();
        }
        finally {
            try {
                if (ais != null) {
                    ais.close();
                }
                if (fis != null) {
                    fis.close();
                }
            }
            catch (Exception e) {
            }

        }
    }
}

Output:

[APPLICATION 14][[CONTEXT 0]5, [CONTEXT 1]14, [CONTEXT 2]#03050020000000, [CONTEXT 3][APPLICATION 1][[CONTEXT 0]5, [CONTEXT 1]ADS.DELTEK.COM, [CONTEXT 2][[CONTEXT 0]2, [CONTEXT 1][HTTP, ashv0135.ads.deltek.com]], [CONTEXT 3][[CONTEXT 0]18, [CONTEXT 1]33, [CONTEXT 2]#c31b7c9979fa6463264b03ba2cc87a2351e828965f005588ba7b0a786e35bad44f26d2399e86d8bcc977cdec6fb9238323552e77ad68ceef8e09ee59e4ba316f7b154de19bbf56f7d0ab246cb20e2d2b9404bfe491b876e2ce8305cd1dc2709af9cf383c11a33dff45b00c0150e9fa8a82cb71f9513340d9984bca72049dd89df52cb51911aa49b620a68f307dc1f2dae284b9de930acd9d10827bddb6add0bdf7dc1619bd99aa6a374b6910104c242f3fe643e0439f30b8d48b3292dcc27f481577df4db9a32940f81f73b1917e67cd6f325d658e0d00456dcab02489401656a157a7747861b8bcd7f1144a1ef3f887d845db6ab843e2b72ce76b845c4b859f3cd2bc72ee96addf5a35fbba0b40fa271d27599ae35402118f2e492406167e26af325b6da41c3c4c9ac74e95e1d804e95fb6c87d08523b73e3556c25641d185bdf339367f63eb08a041e36cac2a09a4a56413077779a1e0041aaca414759293ff6269d7cd8282fd1109d0ed4144277ae8d313dc3b0e281af966013bddccf97b944dd9e5468942adb24cc5302898291ae5ddfe78922fc6c4077e8031c3f310ad06560101dc592855949019003cb71cdef0d75068fc076323fac8bd56a73103eb04d522a67853f9313da0b1cb947900d6f4c7878424cb4ca3a73a65b2cd0b9f92604df7c6340958f5e50f183d1afd54eb6f2d5a344babcdbe65f51898607c97d3d79c62e50da10cb9733413c5cf7d3cf8f534775044061cb98113ee117e768b850c07dbf8ea5d2e657e80fb326568f961ca589f9cb174a487dfefa7b2a18d533755b6e2fc45bffa29119431e1da76e72af27781646c94cd11f0de9acf80982ea447bc13970e7f109927b9035253075a090aecd8c2d68cdd46bfbc71dc0f0b2c849628440ae25fe4049182fe325ae17e8ca88db3dcdc573277eaad871e5549158e3514021049b1f66cb2e2ef1cd597e1c5dae800470521a4f271ba0c8bf235b13038a4391a4288478b15195b142221dda10be36e8d4386f096f77ae80b769acde51216915424591def239ff5873f178a2ef194c9e6604adb41fcdf729c2da86c0aea8c1161dcb8b5278206d377c9db72ed6ff9210048d6d59b87c34ad54612bd56d7530fd542d184d7e5911dbe6f2d873687ffd69439acf556ae30660fa212e54901ec3943bfe5882409fe093ad91e9642a4880ab5457579a21d911ee5b67a41637484f3ea288865ece1022e7b9334b1f4f6ccc06e7ef5b5e1c9cf6f66eb3ed24a287a2d927a5f1a68e5cbdd2d3326437ddc5af2c1f5882d2931162d55fd2e46e61aa4ab3a049f614b6585a96d9c9e0884484dad0e488e59b92ff409f0778a13086e84a2810342407b2e6b299df627b14ff77484f87dce4322fae49f37c3b5a406b535b1cdf083ff528413dfd181b3f5d34da1c7b8effdcf11832d485da8996f3bd8e821bfa5b0e343903cbdcafeeedee1f01a73d99b42c7860a565ca73e8e1924d3c5b0134d5bf71993b6b9e1cd3f98e1c63ada716c785862b6c4e6952176a38ed8f01e5ec2e4bc919fdbc9a976123167da5a229f7c5764889c26735c5624567afc6db5d1374e29580d1b4f0caa7afed6d851df5112066d148a27475c1c4f95b319c4948b35401c749c03a47f6cd47052f41d58ac2077f2de8ac2485f8cc37a355f7f6825acaccdff6a2467ebbd8e291be87e9e74399c6ffb4ac1ab01a3fd44b758b4688bd54177b5724c3d10755d76f0c52a4fec2846f6afcbd8c254443eaec1e6d1505c88695a7d890caf505bcc5a1c8494ab589084c30cd0e2554fbfe91fbca9c44cef1fefd55d95c657e305e6a45b3a0c496a433bb4d11f09a01cb992dd354672c46a9e855077d767a416cd8aea86f70ca4826aa9bbddca24d450cae09e014f21fbde4e383985459d4e8f72f13e440262931fe11e2fc5fb529d7208056267003649655a55ab6da771e472f5d5479b29e64c5205c6ea9bb0c09dde4c9f72f63e17384bfec8cdafd97594ef81974c912da8f4b1409f72d69278d6e119ab055085aaf50bac9cdaaa207a5dcab42844dd4b739af702d78806e307f9a07266d36fc5ba4124e24b9c9f758b1af9d462c35aa0846854f781c7662a720a4fc5e779d517ce709816ed2dcd5a6e6f2273f0f2a82257e9f1f6892a027fd6c12574986a4fc3ce220981b273554ba7b76b7950b581298826ee843ac049a9df92d99d7d95bfd201de46410aafd7db396a2e51a063ba3898e3f65da6d7654b06e2d6ff51f7324358e3ed123795d6c5b166ada71a23f779a92cf1c8a29fa6656de8f5dafb03a481eb6e059c15ddfcee24206be2e671e1b004767a42cb70959d92915478aad2a8d7fa65d30b2db0ef5408fab305c87a91dc4ffc42e0fbede9d1a2009d570688708a4e14ab26cac6533c931d56d65bddffcc23171e529f34f0d5e5c3fbd9244a93afc]], [CONTEXT 4][[CONTEXT 0]18, [CONTEXT 2]#634508da1d066d5824d25cbda9dd0f31b762105087b0ab5618917987c8c7fa1290cab57563aa51d3831bb5e743295bd3a103b4da287dcb83669f1ce6ecede4c3b7166a893e206caeed5725b56945cff800e6761034984dfa7d11d97ea6074d3527f5a7fe8e0e9b9262be166d17becf8af23d4263a0ef67e035b0f670523ca18cb0dc8f089697c7a614dab7403fb271c93846d7fd91afab6803f5aeda420b5cae20662855ba5e0834f2c42da1e49e6d174addc61c60f2291b603fbf01191c2f56539b535482a6b18c7a035fb16df426e0e117d984636250e04a58507292b410868797508c9ebd2ba597c9e11f2a4eaaa583cedf383c62f92312022b3aa2002cc2b6be6211c92a8296964386d01ee1653d2ff99bfda53157905f4a36e7cabc22c6c147c04298ae7f8a85bfcb1d7c7576a94db8b1d6a3db8f95ba1245df874782db002da3f0030d631c5ed74c039c4cb5e6725c67e93c245cee5075f6c7839986378647158b700612d6da39edd055f4dcaf618ee7e62f4bfa409262745385a9304b4df62a6b47c971fa919c9182045efb45ba8952543d1d2428f708410aaeb34bb6bb8e910c965ce7d2639bf2a5826626a7ac]]

The remaining inner message is then fine to parse using the existing parser, as shown above.

KRB5's GSSAPI is manually reading a 16-bit value:

    if (tok_type != -1) {
        if (k5_input_get_uint16_be(&in) != tok_type)
            return in.status ? G_BAD_TOK_HEADER : G_WRONG_TOKID;
    }

(called from e.g. here).

We don't have an incremental ASN.1 parser in this sense, that supports arbitrary non-ASN.1 segments, AFAICT, so we can't read these 2 bytes easily (unless @dghgit knows otherwise; ASN1StreamParser doesn't appear to do what we want as it will still greedily read the outer SEQUENCE and attempt to parse its contents without reference to a spec).

I don't think we have a good example for this in BC right now; X500Name has an inner AttributeTypeAndValue that is of type ANY, but it doesn't include the non-ASN1 token prefix specifier like GSS API does.

[dghgit]

Ah. All I can say is, that's a relief! Thanks for the follow up.

[ivkina]

@cipherboy, @dghgit - thanks a lot, will try the suggestion and post an update accordingly.

As far as Microsoft, this is enterprise licened Windows Server 2022 domain controller, patched to the latest, so, I assume tens of millions of ADs around the world generate billions of tokens like that, in the same format ;)

[cipherboy]

@ivkina I don't think this (my suggestion) actually works; you'll need to check the first byte (0x60), then check the length manually against the rest of the message length (because different tokens will have different lengths) -- which may be 1-3 bytes, see ASN.1 length encoding rules for that -- and then check the OID explicitly via byte array comparison given above, check the inner two-byte thing, and then decode the rest of the data. So its a bit more work than I thought... Sorry :-)

[cipherboy]

@dghgit Hmm, lazy evaluation should exist per source code (I was looking at older javadocs, my bad!):

    /**     
     * Create an ASN1InputStream where no DER object will be longer than limit, and constructed
     * objects such as sequences will be parsed lazily.
     *      
     * @param input stream containing ASN.1 encoded data.
     * @param lazyEvaluate true if parsing inside constructed objects can be delayed.
     */     
    public ASN1InputStream(InputStream input, boolean lazyEvaluate)
    {
        this(input, StreamUtil.findLimit(input), lazyEvaluate);
    }

but this still throws the same assertion:

org.bouncycastle.asn1.ASN1Exception: BOOLEAN value should have 1 byte in it
	at org.bouncycastle.asn1.ASN1InputStream.createPrimitiveDERObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.buildObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readVector(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readVector(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readTaggedObjectDL(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.buildObject(Unknown Source)
	at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source)
	at Main.main(Main.java:20)
Caused by: java.lang.IllegalArgumentException: BOOLEAN value should have 1 byte in it
	at org.bouncycastle.asn1.ASN1Boolean.createPrimitive(Unknown Source)
	... 9 more

I think providing a dedicated parser might be our best bet?

[ivkina]

@cipherboy - I did not try custom code you suggested but based on the last comment I assume the standard asn1 parser will still throw the same error, correct?

[ivkina]

@cipherboy - so, there is no really a simple way unless we want to override ASN1InputStream.readObject(), correct?

[cipherboy]

@ivkina That's my understanding; even then, it is difficult because ASN1TaggedObject doesn't have a lazy variant like ASN1Sequence does (LazyEncodedSequence), and if you were to write that, you'd still need to add helpers to get+skip the two byte from the inner parsing of that field (as otherwise, when you get to that field, you'd read the value and realize it still is an empty boolean).

I've built a custom parser for this and pushed to an internal branch; if @dghgit likes it, we can ship it. But it likely won't make 1.78, 1.79 is more likely. :-)

[ivkina]

@cipherboy - that would be awesome! any chance you can share the parser code? I could test it as well running it against real time ms AD and trying different accounts/tokens.

[cipherboy]

Here's the parser code:

Parser code

package org.bouncycastle.asn1.gssapi;

import java.io.ByteArrayInputStream;
import java.io.EOFException;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;

import org.bouncycastle.asn1.ASN1InputStream;
import org.bouncycastle.asn1.ASN1ObjectIdentifier;
import org.bouncycastle.util.Arrays;

/**
 * This class exists to parse GSS-API Tokens per RFC 4121 Section 4.1
 * Context Establishment Tokens. This is a pseudo-ASN.1 structure given
 * by:
 *
 *         GSS-API DEFINITIONS ::=
 *
 *         BEGIN
 *
 *         MechType ::= OBJECT IDENTIFIER
 *         -- representing Kerberos V5 mechanism
 *
 *         GSSAPI-Token ::=
 *         -- option indication (delegation, etc.) indicated within
 *         -- mechanism-specific token
 *         [APPLICATION 0] IMPLICIT SEQUENCE {
 *                 thisMech MechType,
 *                 innerToken ANY DEFINED BY thisMech
 *                    -- contents mechanism-specific
 *                    -- ASN.1 structure not required
 *                 }
 *
 *   The innerToken field starts with a two-octet token-identifier
 *   (TOK_ID) expressed in big-endian order, followed by a Kerberos
 *   message.
 *
 *          Token               TOK_ID Value in Hex
 *         -----------------------------------------
 *          KRB_AP_REQ            01 00
 *          KRB_AP_REP            02 00
 *          KRB_ERROR             03 00
 *
 * Further details about this structure is given in the earlier RFC 2743
 * Section 3.1 Mechanism-Independent Token Format.
 *
 * Note that this is necessary because while lazy parsing exists in
 * ASN1InputStream, (1) it doesn't work for application-specific
 * tagged objects, and (2) even if we were to implement lazy parsing
 * of tagged objects, we wouldn't be able to skip the two byte header.
 * This means we're better off manually parsing the outer SEQUENCE and
 * its members, until we hit the actual contents of the innerToken.
 *
 * This class makes no attempt to parse GSSAPI-Token values without
 * these two byte headers; use ASN1InputStream directly instead.
 *
 * This parser is re-callable. If a series of concatenated GSSAPI-TOKENs
 * are given in a single InputStream, these can all be parsed by repeatedly
 * calling parse(...) and getting the object out of the tokenInputStream.
 * Note that calling parse again prior to fetching the underlying token object
 * will cause undefined behavior.
 */
public class GSSAPITokenParser
{
    /**
     * innerToken contains KRB_AP_REQUEST.
     */
    public static short KRB_AP_REQ = 0x0100;

    /**
     * innerToken contains KRB_AP_REPLY.
     */
    public static short KRB_AP_REP = 0x0200;

    /**
     * innerToken contains KRB_ERROR.
     */
    public static short KRB_ERROR  = 0x0300;

    /**
     * Create a GSSAPITokenParser based on the specified input stream.
     *
     * @param input stream containing ASN.1 encoded data.
     */
    public GSSAPITokenParser(InputStream is)
    {
        this(is, null, null);
    }

    /**
     * Create a GSSAPITokenParser based on the specified input stream where no
     * object will be larger than the limit.
     *
     * @param input stream containing ASN.1 encoded data.
     * @param limit maximum size of a DER encoded object.
     */
    public GSSAPITokenParser(InputStream is, int limit)
    {
        this(is, limit, null);
    }

    /**
     * Create a GSSAPITokenParser based on the specified input stream where
     * constructed objects such as sequences will be parsed lazily.
     *
     * @param input stream containing ASN.1 encoded data.
     * @param lazyEvaluate true if parsing inside constructed objects can be delayed.
     */
    public GSSAPITokenParser(InputStream is, boolean lazyEvaluate)
    {
        this(is, null, lazyEvaluate);
    }

    /**
     * Create a GSSAPITokenParser based on the specified input stream where no
     * object will be larger than the limit and constructed objects such as
     * sequences will be parsed lazily.
     *
     * @param input stream containing ASN.1 encoded data.
     * @param limit maximum size of a DER encoded object.
     * @param lazyEvaluate true if parsing inside constructed objects can be delayed.
     */
    public GSSAPITokenParser(InputStream is, int limit, boolean lazyEvaluate)
    {
        this(is, (Integer) limit, (Boolean) lazyEvaluate);
    }

    /**
     * Create a GSSAPITokenParser based on the input byte array.
     *
     * @param input array containing ASN.1 encoded data.
     */
    public GSSAPITokenParser(byte[] input)
    {
        this(new ByteArrayInputStream(input), input.length);
    }

    /**
     * Create a GSSAPITokenParser based on the input byte array where
     * constructed objects such as sequences will be parsed lazily.
     *
     * @param input array containing ASN.1 encoded data.
     * @param lazyEvaluate true if parsing inside constructed objects can be delayed.
     */
    public GSSAPITokenParser(byte[] input, boolean lazyEvaluate)
    {
        this(new ByteArrayInputStream(input), input.length, lazyEvaluate);
    }

    // == Implementation ==

    private InputStream is;
    private Integer     limit;
    private Boolean     lazyEvaluate;

    private ASN1ObjectIdentifier thisMech;
    private short                tokenType;
    private ASN1InputStream      tokenInputStream;

    private GSSAPITokenParser(InputStream is, Integer limit, Boolean lazyEvaluate)
    {
        this.is = is;
        this.limit = limit;
        this.lazyEvaluate = lazyEvaluate;
    }

    private void condDecrementLimit(int bytes)
    {
        if (limit == null)
        {
            return;
        }

        limit -= bytes;
    }

    private void condCheckLimit(int bytes)
        throws IOException
    {
        if (bytes < 0)
        {
            throw new IOException("negative bytes to read: " + bytes);
        }

        if (limit == null)
        {
            return;
        }

        if (bytes > limit)
        {
            throw new IOException("wanted to read " + bytes + " > " + limit + " bytes, exceeding limit");
        }
    }

    private void condCheckAndDecrementLimit(int bytes)
        throws IOException
    {
        condCheckLimit(bytes);
        condDecrementLimit(bytes);
    }

    /**
     * Parse a single GSS-API Token from the provided input stream.
     *
     * thisMech and tokenType should be used to inform the parsing of the
     * tokenInputStream; the latter must be consumed prior to calling
     * parse(...) again if multiple GSS-API Tokens are present in the input
     * stream.
     */
    public void parse()
        throws IOException
    {
        thisMech = null;
        tokenType = 0;
        tokenInputStream = null;

        // This manually parses the input stream for the initial tag,
        // consuming it. We then manually parse and validate the length,
        // then the mech oid, and finally the token.

        // 1. Tag
        //
        // Fetch and validate our tag against expectations. RFC 2743 is clear
        // this should be exactly the byte 0x60 and nothing else.
        condCheckAndDecrementLimit(1);
        int sequenceTag = is.read();
        if (sequenceTag != 0x60)
        {
            throw new IOException("invalid tag for [APPLICATION 0] SEQUENCE: " + sequenceTag);
        }

        int sequenceLength = readLength();

        // Validate sequence length fits in expected limit.
        condCheckLimit(sequenceLength);

        // 2. OID
        //
        // Parse the OID. We read the tag and length explicitly, read those
        // bytes, and then parse it as an ASN1ObjectIdentifier explicitly.

        // Validate the OID tag against expectations. RFC 2743 is clear this
        // should be exactly the byte 0x06 and nothing else.
        condCheckAndDecrementLimit(1);
        int oidTag = is.read();
        if (oidTag != 0x06)
        {
            throw new IOException("invalid tag for inner OBJECT IDENTIFIER: " + oidTag);
        }

        int oidLength = readLength();

        // Read the OID now.
        condCheckAndDecrementLimit(oidLength);
        byte[] oidValue = new byte[oidLength];
        int oidReadLength = is.read(oidValue);
        if (oidReadLength != oidLength)
        {
            throw new IOException("read " + oidReadLength + " < " + oidLength + " expected octets in OID");
        }

        // Place the OID into an ASN1ObjectIdentifier.
        thisMech = ASN1ObjectIdentifier.fromContents(oidValue);

        // 3. Token
        //
        // Read two more bytes; this should be the type of the inner contents
        // under RFC 4121.
        condCheckAndDecrementLimit(1);
        int highOrderTokenId = is.read();
        condCheckAndDecrementLimit(1);
        int lowOrderTokenId = is.read();

        tokenType = (short)((highOrderTokenId << 8) | lowOrderTokenId);

        // Now place the remaining input stream in the parser for the token.
        if (limit == null && lazyEvaluate == null)
        {
            tokenInputStream = new ASN1InputStream(is);
        }
        else if (limit == null)
        {
            tokenInputStream = new ASN1InputStream(is, (boolean) lazyEvaluate);
        }
        else if (lazyEvaluate == null)
        {
            tokenInputStream = new ASN1InputStream(is, (int) limit);
        }
        else
        {
            tokenInputStream = new ASN1InputStream(is, (int) limit, (boolean) lazyEvaluate);
        }
    }

    /**
     * Return thisMech from the GSS-API Token description.
     */
    public ASN1ObjectIdentifier getThisMech()
    {
        return thisMech;
    }

    /**
     * Returns the tokenType from the GSS-API token description.
     */
    public short getTokenType()
    {
        return tokenType;
    }

    /**
     * Returns the innerToken ASN.1 stream (after 2-byte tokenType).
     */
    public ASN1InputStream getTokenInputStream()
    {
        return tokenInputStream;
    }

    private int readLength() throws IOException
    {
        // Compute the length of this segment.
        int result = 0;

        condCheckAndDecrementLimit(1);
        int lengthByte = is.read();
        if (lengthByte < 127)
        {
            // 2.a: length fits in a single byte; definite length.
            result = (int)lengthByte;
        }
        else
        {
            // 2.b: Bits 1..7 are the length of the length; the remaining
            // bytes are read and validated.
            int numOctets = lengthByte & 0x7f;
            if (numOctets > 4)
            {
                throw new IOException(numOctets + " bytes of length is excessively long");
            }
            else if (numOctets == 0)
            {
                throw new IOException("got zero for multi-octet length encoding; refusing to parse indefinite length token");
            }

            condCheckAndDecrementLimit(numOctets);
            byte[] rawLength = new byte[numOctets];
            int readLength = is.read(rawLength);
            if (readLength != numOctets)
            {
                throw new IOException("read " + readLength + " < " + numOctets + " expected octets in length encoding");
            }

            // There cannot be any leading zeros. Direct zero indexing is
            // fine as we know numOctets > 0, readLength == numOctets and
            // thus we have at least one element here.
            if (rawLength[0] == 0x00)
            {
                throw new IOException("malformed length: has leading zeros");
            }

            for (byte rawLengthByte : rawLength)
            {
                result = (result << 8) + rawLengthByte;
                if ((result >>> 23) != 0)
                {
                    // check preserved from readLength() of ASN1InputStream
                    throw new IOException("long form definite-length more than 2^23 bits");
                }
            }
        }

        return result;
    }
}

Let me know if it works for you! Not sure I'm quite happy with the inner parsing yet, still gotta look around to see if I can reuse some things.

[ivkina]

@cipherboy - thanks! will give it a try next week and will post an update

[ivkina]

@cipherboy - I think I got it working with your parser...-:) again, many, many thanks for such quick turnaround, really appreaciate that!

One comment - the overall model/hierarchy of asn1 objects changed quite a bit between versions 1.60 and 1.71. everything seems to be built around ASN1TaggedObject... however, couple of places where this was not the case. see line 64 - 70 in the code in the very bottom where it took me to call tagged.getBaseObject() first to get to the correct ASN1TaggedObject. Otherwise, it returns deprecated DLApplicationSpecific instance as a tagged object and then failed on .getBaseUniversal(...). Why is that?

// this line throws java.lang.IllegalStateException: unexpected object: org.bouncycastle.asn1.DLApplicationSpecific
// ASN1Sequence seq = (ASN1Sequence) tagged.getBaseUniversal(true, BERTags.SEQUENCE);
// but this line works - what is the right way to use tagged object here??? why do I need to call getBaseObject() intermediary ???
ASN1Sequence seq = (ASN1Sequence) ((ASN1TaggedObject) tagged.getBaseObject()).getBaseUniversal(true, BERTags.SEQUENCE);

another example was when ASN1Sequence.getObjects() returned enumeration of DERGeneralString objects whereas I was expecting ASN1TaggedObject instances only...what is the criteria? see lines 88-91 in the code below.

you can run the code using the same krb.txt attached to the case.

package com;

import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.math.BigInteger;
import java.security.Security;
import java.util.Enumeration;

import org.bouncycastle.asn1.ASN1GeneralString;
import org.bouncycastle.asn1.ASN1InputStream;
import org.bouncycastle.asn1.ASN1Integer;
import org.bouncycastle.asn1.ASN1ObjectIdentifier;
import org.bouncycastle.asn1.ASN1OctetString;
import org.bouncycastle.asn1.ASN1Sequence;
import org.bouncycastle.asn1.ASN1TaggedObject;
import org.bouncycastle.asn1.BERTags;
import org.bouncycastle.asn1.DERBitString;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

public class TestASN1 {

	public static void main(String[] args) {
		File f = new File("c:/temp/krb.txt");
		FileInputStream fis = null;
		DataInputStream dis = null;
		ASN1InputStream ais = null;
		try {
			Security.addProvider(new BouncyCastleProvider());

			fis = new FileInputStream(f);
			dis = new DataInputStream(fis);
			byte[] token = new byte[(int) f.length()];
			dis.readFully(token);

			ais = toASN1Stream(token);
			ASN1TaggedObject to = (ASN1TaggedObject) ais.readObject();
			ais.close();

			Enumeration<ASN1TaggedObject> e = ((ASN1Sequence) to.getBaseUniversal(true, BERTags.SEQUENCE)).getObjects();
			while (e.hasMoreElements()) {
				ASN1TaggedObject tagged = e.nextElement();
				switch (tagged.getTagNo()) {
				case 0:
					ASN1Integer pvno = (ASN1Integer) tagged.getBaseUniversal(true, BERTags.INTEGER);
					if (!pvno.getValue().equals(new BigInteger("5"))) {
						throw new Exception("Kerberos version invalid - " + pvno);
					}
					break;
				case 1:
					ASN1Integer msgType = (ASN1Integer) tagged.getBaseUniversal(true, BERTags.INTEGER);
					if (!msgType.getValue().equals(new BigInteger("14"))) {
						throw new Exception("Kerberos request invalid - " + msgType.getValue());
					}
					break;
				case 2:
					DERBitString bitString = (DERBitString) tagged.getBaseUniversal(true, BERTags.BIT_STRING);
					// byte apOptions = bitString.getBytes()[0];
					break;
				case 3:
					// this line throws java.lang.IllegalStateException: unexpected object: org.bouncycastle.asn1.DLApplicationSpecific

					// ASN1Sequence seq = (ASN1Sequence) tagged.getBaseUniversal(true, BERTags.SEQUENCE);

					// but this line works - what is the right way to use tagged object??? why do I need to call getBaseObject() intermediary ???
					ASN1Sequence seq = (ASN1Sequence) ((ASN1TaggedObject) tagged.getBaseObject()).getBaseUniversal(true, BERTags.SEQUENCE);
					Enumeration<ASN1TaggedObject> e2 = seq.getObjects();
					while (e2.hasMoreElements()) {
						tagged = e2.nextElement();
						switch (tagged.getTagNo()) {
						case 0:
							// version
							pvno = (ASN1Integer) tagged.getBaseUniversal(true, BERTags.INTEGER);
							break;
						case 1:
							// spn realm
							ASN1GeneralString realm = (ASN1GeneralString) tagged.getBaseUniversal(true, BERTags.GENERAL_STRING);
							break;
						case 2:
							// spn name
							ASN1TaggedObject taggedSpn = (ASN1TaggedObject) ((ASN1Sequence) tagged.getBaseUniversal(true, BERTags.SEQUENCE)).getObjectAt(1);
							Enumeration e3 = ((ASN1Sequence) taggedSpn.getBaseUniversal(true, BERTags.SEQUENCE)).getObjects();
							StringBuilder sb = new StringBuilder();
							while (e3.hasMoreElements()) {
								// not ASN1TaggedObject but DERGeneralString
								// ASN1TaggedObject taggedPart = e3.nextElement();
								// sb.append(((ASN1GeneralString) taggedPart.getBaseUniversal(true, BERTags.GENERAL_STRING)).getString() + "/");
								sb.append(((ASN1GeneralString) e3.nextElement()).getString() + "/");
							}
							sb.setLength(sb.length() - 1);
							break;
						case 3:
							// encrypted part
							ASN1Sequence encSeq = ((ASN1Sequence) tagged.getBaseUniversal(true, BERTags.SEQUENCE));
							ASN1Integer encType = (ASN1Integer) ((ASN1TaggedObject) encSeq.getObjectAt(0)).getBaseUniversal(true, BERTags.INTEGER);
							ASN1OctetString encOctets = (ASN1OctetString) ((ASN1TaggedObject) encSeq.getObjectAt(2)).getBaseUniversal(true, BERTags.OCTET_STRING);
							byte[] cipher = encOctets.getOctets();
							// run decrypt...
							
							break;
						}
					}
					break;
				case 4:
					break;
				default:
					throw new Exception("Kerberos field invalid - " + tagged.getTagNo());
				}
			}

		}
		catch (Exception e) {
			e.printStackTrace();
		}
		finally {
			try {
				if (ais != null) {
					ais.close();
				}
				if (dis != null) {
					dis.close();
				}
				if (fis != null) {
					fis.close();
				}
			}
			catch (Exception e) {
			}

		}
	}

	/**
	 * innerToken contains KRB_AP_REQUEST.
	 */
	public static short KRB_AP_REQ = 0x0100;

	/**
	 * innerToken contains KRB_AP_REPLY.
	 */
	public static short KRB_AP_REP = 0x0200;

	/**
	 * innerToken contains KRB_ERROR.
	 */
	public static short KRB_ERROR = 0x0300;

	public static ASN1InputStream toASN1Stream(byte[] token) throws Exception {
		KerberosTokenParser ktp = (new TestASN1()).new KerberosTokenParser(new ByteArrayInputStream(token));
		try {
			ktp.parse();
		}
		catch (IOException e) {
			throw new Exception("Failed to parse kerberos token.", e);
		}
		return ktp.getTokenInputStream();
	}

	class KerberosTokenParser {

		/**
		 * Create a KerberosTokenParser based on the specified input stream.
		 *
		 * @param input
		 *          stream containing ASN.1 encoded data.
		 */
		public KerberosTokenParser(InputStream is) {
			this(is, null, null);
		}

		/**
		 * Create a KerberosTokenParser based on the specified input stream where no object will be larger than the limit.
		 *
		 * @param input
		 *          stream containing ASN.1 encoded data.
		 * @param limit
		 *          maximum size of a DER encoded object.
		 */
		public KerberosTokenParser(InputStream is, int limit) {
			this(is, limit, null);
		}

		/**
		 * Create a KerberosTokenParser based on the specified input stream where constructed objects such as sequences will be parsed lazily.
		 *
		 * @param input
		 *          stream containing ASN.1 encoded data.
		 * @param lazyEvaluate
		 *          true if parsing inside constructed objects can be delayed.
		 */
		public KerberosTokenParser(InputStream is, boolean lazyEvaluate) {
			this(is, null, lazyEvaluate);
		}

		/**
		 * Create a KerberosTokenParser based on the specified input stream where no object will be larger than the limit and constructed objects such as sequences
		 * will be parsed lazily.
		 *
		 * @param input
		 *          stream containing ASN.1 encoded data.
		 * @param limit
		 *          maximum size of a DER encoded object.
		 * @param lazyEvaluate
		 *          true if parsing inside constructed objects can be delayed.
		 */
		public KerberosTokenParser(InputStream is, int limit, boolean lazyEvaluate) {
			this(is, (Integer) limit, (Boolean) lazyEvaluate);
		}

		/**
		 * Create a KerberosTokenParser based on the input byte array.
		 *
		 * @param input
		 *          array containing ASN.1 encoded data.
		 */
		public KerberosTokenParser(byte[] input) {
			this(new ByteArrayInputStream(input), input.length);
		}

		/**
		 * Create a KerberosTokenParser based on the input byte array where constructed objects such as sequences will be parsed lazily.
		 *
		 * @param input
		 *          array containing ASN.1 encoded data.
		 * @param lazyEvaluate
		 *          true if parsing inside constructed objects can be delayed.
		 */
		public KerberosTokenParser(byte[] input, boolean lazyEvaluate) {
			this(new ByteArrayInputStream(input), input.length, lazyEvaluate);
		}

		// == Implementation ==

		private InputStream is;
		private Integer limit;
		private Boolean lazyEvaluate;

		private ASN1ObjectIdentifier thisMech;
		private short tokenType;
		private ASN1InputStream tokenInputStream;

		private KerberosTokenParser(InputStream is, Integer limit, Boolean lazyEvaluate) {
			this.is = is;
			this.limit = limit;
			this.lazyEvaluate = lazyEvaluate;
		}

		private void condDecrementLimit(int bytes) {
			if (limit == null) {
				return;
			}

			limit -= bytes;
		}

		private void condCheckLimit(int bytes)
				throws IOException {
			if (bytes < 0) {
				throw new IOException("negative bytes to read: " + bytes);
			}

			if (limit == null) {
				return;
			}

			if (bytes > limit) {
				throw new IOException("wanted to read " + bytes + " > " + limit + " bytes, exceeding limit");
			}
		}

		private void condCheckAndDecrementLimit(int bytes)
				throws IOException {
			condCheckLimit(bytes);
			condDecrementLimit(bytes);
		}

		/**
		 * Parse a single GSS-API Token from the provided input stream.
		 *
		 * thisMech and tokenType should be used to inform the parsing of the tokenInputStream; the latter must be consumed prior to calling parse(...) again if
		 * multiple GSS-API Tokens are present in the input stream.
		 */
		public void parse()
				throws IOException {
			thisMech = null;
			tokenType = 0;
			tokenInputStream = null;

			// This manually parses the input stream for the initial tag,
			// consuming it. We then manually parse and validate the length,
			// then the mech oid, and finally the token.

			// 1. Tag
			//
			// Fetch and validate our tag against expectations. RFC 2743 is clear
			// this should be exactly the byte 0x60 and nothing else.
			condCheckAndDecrementLimit(1);
			int sequenceTag = is.read();
			if (sequenceTag != 0x60) {
				throw new IOException("invalid tag for [APPLICATION 0] SEQUENCE: " + sequenceTag);
			}

			int sequenceLength = readLength();

			// Validate sequence length fits in expected limit.
			condCheckLimit(sequenceLength);

			// 2. OID
			//
			// Parse the OID. We read the tag and length explicitly, read those
			// bytes, and then parse it as an ASN1ObjectIdentifier explicitly.

			// Validate the OID tag against expectations. RFC 2743 is clear this
			// should be exactly the byte 0x06 and nothing else.
			condCheckAndDecrementLimit(1);
			int oidTag = is.read();
			if (oidTag != 0x06) {
				throw new IOException("invalid tag for inner OBJECT IDENTIFIER: " + oidTag);
			}

			int oidLength = readLength();

			// Read the OID now.
			condCheckAndDecrementLimit(oidLength);
			byte[] oidValue = new byte[oidLength];
			int oidReadLength = is.read(oidValue);
			if (oidReadLength != oidLength) {
				throw new IOException("read " + oidReadLength + " < " + oidLength + " expected octets in OID");
			}

			// Place the OID into an ASN1ObjectIdentifier.
			thisMech = ASN1ObjectIdentifier.fromContents(oidValue);

			// 3. Token
			//
			// Read two more bytes; this should be the type of the inner contents
			// under RFC 4121.
			condCheckAndDecrementLimit(1);
			int highOrderTokenId = is.read();
			condCheckAndDecrementLimit(1);
			int lowOrderTokenId = is.read();

			tokenType = (short) ((highOrderTokenId << 8) | lowOrderTokenId);

			// Now place the remaining input stream in the parser for the token.
			if (limit == null && lazyEvaluate == null) {
				tokenInputStream = new ASN1InputStream(is);
			}
			else if (limit == null) {
				tokenInputStream = new ASN1InputStream(is, (boolean) lazyEvaluate);
			}
			else if (lazyEvaluate == null) {
				tokenInputStream = new ASN1InputStream(is, (int) limit);
			}
			else {
				tokenInputStream = new ASN1InputStream(is, (int) limit, (boolean) lazyEvaluate);
			}
		}

		/**
		 * Return thisMech from the GSS-API Token description.
		 */
		public ASN1ObjectIdentifier getThisMech() {
			return thisMech;
		}

		/**
		 * Returns the tokenType from the GSS-API token description.
		 */
		public short getTokenType() {
			return tokenType;
		}

		/**
		 * Returns the innerToken ASN.1 stream (after 2-byte tokenType).
		 */
		public ASN1InputStream getTokenInputStream() {
			return tokenInputStream;
		}

		private int readLength() throws IOException {
			// Compute the length of this segment.
			int result = 0;

			condCheckAndDecrementLimit(1);
			int lengthByte = is.read();
			if (lengthByte < 127) {
				// 2.a: length fits in a single byte; definite length.
				result = (int) lengthByte;
			}
			else {
				// 2.b: Bits 1..7 are the length of the length; the remaining
				// bytes are read and validated.
				int numOctets = lengthByte & 0x7f;
				if (numOctets > 4) {
					throw new IOException(numOctets + " bytes of length is excessively long");
				}
				else if (numOctets == 0) {
					throw new IOException("got zero for multi-octet length encoding; refusing to parse indefinite length token");
				}

				condCheckAndDecrementLimit(numOctets);
				byte[] rawLength = new byte[numOctets];
				int readLength = is.read(rawLength);
				if (readLength != numOctets) {
					throw new IOException("read " + readLength + " < " + numOctets + " expected octets in length encoding");
				}

				// There cannot be any leading zeros. Direct zero indexing is
				// fine as we know numOctets > 0, readLength == numOctets and
				// thus we have at least one element here.
				if (rawLength[0] == 0x00) {
					throw new IOException("malformed length: has leading zeros");
				}

				for (byte rawLengthByte : rawLength) {
					result = (result << 8) + rawLengthByte;
					if ((result >>> 23) != 0) {
						// check preserved from readLength() of ASN1InputStream
						throw new IOException("long form definite-length more than 2^23 bits");
					}
				}
			}

			return result;
		}
	}

}

---

edited by @cipherboy for formatting.