Disable all parsers and processors by default

[markkolich]

I have a need to disable all parsers and processors by default, while selectively enabling only a fixed subset of supported features. The use-case I need to address is "only support ** and _" , while disabling all other Markdown syntax features.

For instance:

final MutableDataSet options = new MutableDataSet();

// Disable everything I don't need; not sure this is "everything"...
options.set(Parser.BLOCK_QUOTE_PARSER, false);
options.set(Parser.FENCED_CODE_BLOCK_PARSER, false);
options.set(Parser.HEADING_PARSER, false);
options.set(Parser.HTML_BLOCK_DEEP_PARSER, false);
options.set(Parser.HTML_BLOCK_PARSER, false);
options.set(Parser.INDENTED_CODE_BLOCK_PARSER, false);
options.set(Parser.LIST_BLOCK_PARSER, false);
options.set(Parser.THEMATIC_BREAK_PARSER, false);
options.set(Parser.REFERENCE_PARAGRAPH_PRE_PROCESSOR, false);

// Then, only enable a tiny subset of processors I need.
options.set(Parser.ASTERISK_DELIMITER_PROCESSOR, true);
options.set(Parser.UNDERSCORE_DELIMITER_PROCESSOR, true);

This works fine, afaict, but I'm left to enumerate all of the parsers and processors I can find and disable them.

The documentation at https://github.com/vsch/flexmark-java/wiki/Extensions#parser says:

Unified options handling added which are also used to selectively disable loading of core parsers and processors.

Is there a convenient way for me to "go the other way" and disable everything by default while only selectively enabling the features I need?

It would be great if the "feature enablement" configuration DataKey's were split into their own class so it's at least obvious what needs to be disabled if one intends to turn off a "feature".

((I've considered implementing my own DataHolder and the corresponding getOrCompute method therein to blindly return "false" on any DataKey I don't recognize. That of course, only works for boolean keys.))

CC // @larrysteinke

[vsch]

@markkolich, at the moment there are no option to disable individual inline parser elements, other than a few exceptions like the delimiters, since inline parsing is implemented as one big chunk of code with many interdependencies which would be a nightmare to debug and test if some were allowed to be disabled. It would also pepper the code with additional if/else blocks making it error prone and harder to maintain.

The data keys you are interested in are all in the Parser since others are in extensions or the HtmlRenderer. The latter works with whatever nodes are in the AST and extensions are not included by default.

If you disable everything then there is no markdown parsing left and all you have is text with ** and _ markup. It would seem to be simple enough to create a simple parser to handle these without the need for the rest of the library.

As an alternative you can either create custom node renderer that will render other special elements as their plain text content, which effectively removes their specialness out of rendered output.

A better alternative is to create a node post processor which will go through the AST and replace all inline nodes which you do not want with a Text node having the characters of the node you are replacing. This AST can then be rendered by the library renderers for HTML, DOCX or PDF.

I have not put effort into options for disabling everything because it is for limited and rare use cases. I do not want to complicate the code base more than necessary thus adding more maintenance to my plate which is not justified by special cases which can and should be addressed by custom code.

[markkolich]

Thanks for the followup, @vsch.

which would be a nightmare to debug and test if some were allowed to be disabled. It would also pepper the code with additional if/else blocks making it error prone and harder to maintain.

Understood.

It would seem to be simple enough to create a simple parser to handle these without the need for the rest of the library.

I considered that, but given much of the initial legwork has already been done in the form of this library, I'd rather leverage as much as I can as opposed to writing & maintaining additional custom code. I already bumped up against a need to support + for underlined text (HTML rendered using an <ins> tag), to which I've conveniently written my own UnderlineExtension following your examples:

options.set(Parser.ASTERISK_DELIMITER_PROCESSOR, true);
options.set(Parser.UNDERSCORE_DELIMITER_PROCESSOR, true);

options.set(Parser.EXTENSIONS, Arrays.asList(UnderlineExtension.create()));

This works beautifully.

If you disable everything then there is no markdown parsing left and all you have is text with ** and _ markup.

Right, I just wanted to build an options object that effectively "disables everything" to start and then subsequently enable only the parsing/processor features I need. I wholeheartedly understand that in doing so, other Markdown syntax/features won't "render" and simply present themselves as raw tokens (>, #, etc.) in the rendered output which is, in fact, exactly what I want. My use-case only requires support for **, _, and + (for underlined text) and nothing else — as such I want to strictly ignore other Markdown tokens.

As an alternative you can either create custom node renderer that will render other special elements as their plain text content, which effectively removes their specialness out of rendered output.

I'm going to "disable" as much as I can and leave it at that, unless your expert knowledge here strongly advises against this approach.

That said, is what I presented earlier sufficient to disable the "primary" parser/processor features?

options.set(Parser.BLOCK_QUOTE_PARSER, false);
options.set(Parser.FENCED_CODE_BLOCK_PARSER, false);
options.set(Parser.HEADING_PARSER, false);
options.set(Parser.HTML_BLOCK_DEEP_PARSER, false);
options.set(Parser.HTML_BLOCK_PARSER, false);
options.set(Parser.INDENTED_CODE_BLOCK_PARSER, false);
options.set(Parser.LIST_BLOCK_PARSER, false);
options.set(Parser.THEMATIC_BREAK_PARSER, false);
options.set(Parser.REFERENCE_PARAGRAPH_PRE_PROCESSOR, false);
// ... other import ones I missed?

Thanks for the help & consideration.

[vsch]

@markkolich, disabling the block parsers is stable and will work.

Disabling delimiters and reference paragraph pre-processor will also work. The latter will also disable rendering of all ref links and ref images which is what you want.

Wiki links are disabled by default so this is not an issue.

The real issues are with disabling link [text](url), image link ![text](url), inline code (text in back ticks). These can be easily handled by a node post processor to replace the specific nodes with just text. I will add a code sample to the flexmark-java-samples module to show how to do that.

There are also hard line breaks (two or more spaces before an EOL in paragraphs) but these can be addressed by setting the desired rendering string for these. Default is <br> but you can put an empty string so they don't render.

Other than that you should be good to go.

[markkolich]

@vsch thanks for the tip on the NodePostProcessor!

I was kicking the tires on it, and at least for the replacing of [foobar](http://example.com) with just foobar this seems to work:

import com.vladsch.flexmark.ast.Document;
import com.vladsch.flexmark.ast.Link;
import com.vladsch.flexmark.ast.Node;
import com.vladsch.flexmark.ast.Text;
import com.vladsch.flexmark.parser.block.NodePostProcessor;
import com.vladsch.flexmark.parser.block.NodePostProcessorFactory;
import com.vladsch.flexmark.util.NodeTracker;
import com.vladsch.flexmark.util.options.DataHolder;

public class AnchorLinkReplacerPostProcessor extends NodePostProcessor {

    public AnchorLinkReplacerPostProcessor(DataHolder options) { }

    @Override
    public void process(NodeTracker state, Node node) {
        if (node instanceof Link) {
            final Link link = (Link) node;

            final Text text = new Text(link.getText());
            final Node previous = node.getPrevious();
            if (previous != null) {
                previous.insertAfter(text);
            } else {
                final Node parent = node.getParent();
                parent.prependChild(text);
            }

            node.unlink();
            state.nodeRemoved(node);
        }
    }

    public static class Factory extends NodePostProcessorFactory {
        public Factory(DataHolder options) {
            super(false);

            addNodes(Link.class);
        }

        @Override
        public NodePostProcessor create(Document document) {
            return new AnchorLinkReplacerPostProcessor(document);
        }
    }

}

[foobar](http://example.com) -> foobar

I can extend this for the remaining cases as well.

If you'd like a PR, I'm happy to contribute my resulting example NodePostProcessor's to the flexmark-java-samples package.

[vsch]

@markkolich, for replacing a node you can just do link.insertAfter(text) or link.insertBefore(text) before unlinking the node. Both have the same effect since the link node is unlinked afterwards. No need to check previous or use parent node since the node you are replacing already specifies the insertion point.

You should also do a state.nodeAdded(text) just to keep up house keeping.

Other than that it is quite easy to handle the other nodes and gives you the flexibility of replacement.

[markkolich]

for replacing a node you can just do link.insertAfter(text) or link.insertBefore(text) before unlinking the node.

Ah, right, that's a lot better — thanks @vsch!

Looks like I've got everything working. Per your advice, I implemented a custom "SanitizerPostProcessor" that strips out and gracefully replaces elements of type Link, Image, and Code. I've also got a custom UnderlineExtension wired up that adds underline (<ins>) support for tokens wrapped with +.

I'll clean this up a bit and submit a PR shortly for flexmark-java-samples.

[vsch]

@markkolich, I am glad you decided to kick the tires instead of waiting on me. I will add your sample when you make it available.

This can even be enhanced into a generic post-processor extension with extension options selecting which inline nodes to replace with their text content or some other part of their element's text, like you did for links. It can then be used by others to disable individual inline elements which cannot be disabled at parse time.

Thank you for persisting in the discussion to help arrive at a clean, re-usable solution which does not require chewing up the inline parser. Previous requests for a similar feature ended shortly after asking/expecting me to do the work. These tend to discourage my interest and participation. I am swamped with OSS projects, supported after work hours. I have to prioritize where I invest my effort.