processing.mode.java.preproc

Class PdePreprocessor

java.lang.Object

processing.mode.java.preproc.PdePreprocessor

public class PdePreprocessor
extends Object

Class that orchestrates preprocessing p5 syntax into straight Java.

Current Preprocessor Subsitutions:

• any function not specified as being protected or private will be made 'public'. this means that void setup() becomes public void setup(). This is important to note when coding with core.jar outside of the PDE.
• compiler.substitute_floats (currently "substitute_f") treat doubles as floats, i.e. 12.3 becomes 12.3f so that people don't have to add f after their numbers all the time since it's confusing for beginners.
• compiler.enhanced_casting byte(), char(), int(), float() works for casting. this is basic in the current implementation, but should be expanded as described above. color() works similarly to int(), however there is also a *function* called color(r, g, b) in p5.
• compiler.color_datatype 'color' is aliased to 'int' as a datatype to represent ARGB packed into a single int, commonly used in p5 for pixels[] and other color operations. this is just a search/replace type thing, and it can be used interchangeably with int.
• compiler.web_colors (currently "inline_web_colors") color c = #cc0080; should unpack to 0xffcc0080 (the ff at the top is so that the color is opaque), which is just an int.

Other preprocessor functionality

• detects what 'mode' the program is in: static (no function brackets at all, just assumes everything is in draw), active (setup plus draw or loop), and java mode (full java support). http://processing.org/reference/environment/

The PDE Preprocessor is based on the Java Grammar that comes with ANTLR 2.7.2. Moving it forward to a new version of the grammar shouldn't be too difficult.

Here's some info about the various files in this directory:

java.g: this is the ANTLR grammar for Java 1.3/1.4 from the ANTLR distribution. It is in the public domain. The only change to this file from the original this file is the uncommenting of the clauses required to support assert().

java.tree.g: this describes the Abstract Syntax Tree (AST) generated by java.g. It is only here as a reference for coders hacking on the preprocessor, it is not built or used at all. Note that pde.g overrides some of the java.g rules so that in PDE ASTs, there are a few minor differences. Also in the public domain.

pde.g: this is the grammar and lexer for the PDE language itself. It subclasses the java.g grammar and lexer. There are a couple of overrides to java.g that I hope to convince the ANTLR folks to fold back into their grammar, but most of this file is highly specific to PDE itself. PdeEmitter.java: this class traverses the AST generated by the PDE Recognizer, and emits it as Java code, doing any necessary transformations along the way. It is based on JavaEmitter.java, available from antlr.org, written by Andy Tripp , who has given permission for it to be distributed under the GPL.

ExtendedCommonASTWithHiddenTokens.java: this adds a necessary initialize() method, as well as a number of methods to allow for XML serialization of the parse tree in a such a way that the hidden tokens are visible. Much of the code is taken from the original CommonASTWithHiddenTokens class. I hope to convince the ANTLR folks to fold these changes back into that class so that this file will be unnecessary.

TokenStreamCopyingHiddenTokenFilter.java: this class provides TokenStreamHiddenTokenFilters with the concept of tokens which can be copied so that they are seen by both the hidden token stream as well as the parser itself. This is useful when one wants to use an existing parser (like the Java parser included with ANTLR) that throws away some tokens to create a parse tree which can be used to spit out a copy of the code with only minor modifications. Partially derived from ANTLR code. I hope to convince the ANTLR folks to fold this functionality back into ANTLR proper as well.

whitespace_test.pde: a torture test to ensure that the preprocessor is correctly preserving whitespace, comments, and other hidden tokens correctly. See the comments in the code for details about how to run the test.

All other files in this directory are generated at build time by ANTLR itself. The ANTLR manual goes into a fair amount of detail about the what each type of file is for.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PdePreprocessor.Mode

Field Summary

Fields

Modifier and Type	Field and Description
protected String	indent
protected static LinkedHashMap<String,Pattern>	matchPatterns
protected PdePreprocessor.Mode	mode
protected static String	UNICODE_ESCAPES

Constructor Summary

Constructors

Constructor and Description
PdePreprocessor(String sketchName)
PdePreprocessor(String sketchName, int tabSize)

Method Summary

Modifier and Type	Method and Description
void	addMethod(String methodName)
void	debugAST(antlr.collections.AST ast, boolean includeHidden)
protected static MatchResult	findInCurrentScope(Pattern pattern, CharSequence code) Calls findInScope on the whole string with min and max target scopes set to zero.
protected static MatchResult	findInCurrentScope(Pattern pattern, CharSequence code, int start) Calls findInScope starting at start char with min and max target scopes set to zero.
protected static MatchResult	findInScope(Pattern pattern, CharSequence code, int start, int stop, int minTargetScopeDepth, int maxTargetScopeDepth) Looks for the pattern at a specified target scope depth relative to the scope depth of the starting position.
String[]	getCoreImports()
String[]	getDefaultImports()
boolean	hasMethod(String methodName)
boolean	ignoreImport(String pkg) Return true if this import should be removed from the code.
SurfaceInfo	initSketchSize(String code, boolean sizeWarning)
protected static String[]	matchDensityMess(String searchArea)
protected static String[]	matchMethod(String methodName, String searchArea) Looks for the specified method in the base scope of the search area.
static PdePreprocessor.Mode	parseMode(CharSequence code) Parses the code and determines the mode of the sketch.
static SurfaceInfo	parseSketchSize(String code, boolean fussy) Parse a chunk of code and extract the size() command and its contents.
protected static int	scopeDepthDiff(CharSequence code, int start, int stop) Walks the specified region (not including stop) and determines difference in scope depth.
static String	scrubComments(String what) Replace all commented portions of a given String as spaces.
void	setAdvClassName(String advClassName)
void	setMode(PdePreprocessor.Mode mode)
PreprocessorResult	write(Writer out, String program)
PreprocessorResult	write(Writer out, String program, StringList codeFolderPackages)
protected void	writeDeclaration(PrintWriter out, String className) Write any required header material (eg imports, class decl stuff)
protected void	writeFooter(PrintWriter out, String className) Write any necessary closing text.
protected int	writeImportList(PrintWriter out, List<String> imports)
protected int	writeImportList(PrintWriter out, String[] imports)
protected int	writeImports(PrintWriter out, List<String> programImports, List<String> codeFolderImports)
protected void	writeParseTree(String filename, antlr.collections.AST ast)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

UNICODE_ESCAPES

protected static final String UNICODE_ESCAPES

See Also:

Constant Field Values

indent

protected final String indent

mode

protected PdePreprocessor.Mode mode

matchPatterns

protected static LinkedHashMap<String,Pattern> matchPatterns

Constructor Detail

PdePreprocessor

public PdePreprocessor(String sketchName)

PdePreprocessor

public PdePreprocessor(String sketchName,
                       int tabSize)

Method Detail

initSketchSize

public SurfaceInfo initSketchSize(String code,
                                  boolean sizeWarning)
                           throws SketchException

Throws:

SketchException

parseSketchSize

public static SurfaceInfo parseSketchSize(String code,
                                          boolean fussy)
                                   throws SketchException

Parse a chunk of code and extract the size() command and its contents. Also goes after fullScreen(), smooth(), and noSmooth().

Parameters:

code - The code from the main tab in the sketch

fussy - true if it should show an error message if bad size()

Returns:

null if there was an error, otherwise an array (might contain some/all nulls)

Throws:

SketchException

parseMode

public static PdePreprocessor.Mode parseMode(CharSequence code)

Parses the code and determines the mode of the sketch.

Parameters:

code - code without comments

Returns:

determined mode

findInCurrentScope

protected static MatchResult findInCurrentScope(Pattern pattern,
                                                CharSequence code)

Calls findInScope on the whole string with min and max target scopes set to zero.

findInCurrentScope

protected static MatchResult findInCurrentScope(Pattern pattern,
                                                CharSequence code,
                                                int start)

Calls findInScope starting at start char with min and max target scopes set to zero.

findInScope

protected static MatchResult findInScope(Pattern pattern,
                                         CharSequence code,
                                         int start,
                                         int stop,
                                         int minTargetScopeDepth,
                                         int maxTargetScopeDepth)

Looks for the pattern at a specified target scope depth relative to the scope depth of the starting position. Example: Calling this with starting position inside a method body and target depth 0 would search only in the method body, while using target depth -1 would look only in the body of the enclosing class (but not in any methods of the class or outside of the class). By using a scope range, you can e.g. search in the whole class including bodies of methods and inner classes.

Parameters:

pattern - matching is realized by find() method of this pattern

code - Java code without comments

start - starting position in the code String (inclusive)

stop - ending position in the code Sting (exclusive)

minTargetScopeDepth - desired min scope depth of the match relative to the scope of the starting position

maxTargetScopeDepth - desired max scope depth of the match relative to the scope of the starting position

Returns:

first match at a desired relative scope depth, null if there isn't one

scopeDepthDiff

protected static int scopeDepthDiff(CharSequence code,
                                    int start,
                                    int stop)

Walks the specified region (not including stop) and determines difference in scope depth. Adds one to depth on opening curly brace, subtracts one from depth on closing curly brace. Ignores string and char literals.

Parameters:

code - code without comments

start - start of the region, must not be in string literal, char literal or second char of escaped sequence

stop - end of the region (exclusive)

Returns:

scope depth difference between start and stop, Integer.MAX_VALUE if end is in string literal, char literal or second char of escaped sequence

matchMethod

protected static String[] matchMethod(String methodName,
                                      String searchArea)

Looks for the specified method in the base scope of the search area.

matchDensityMess

protected static String[] matchDensityMess(String searchArea)

scrubComments

public static String scrubComments(String what)

Replace all commented portions of a given String as spaces. Utility function used here and in the preprocessor.

addMethod

public void addMethod(String methodName)

hasMethod

public boolean hasMethod(String methodName)

setAdvClassName

public void setAdvClassName(String advClassName)

setMode

public void setMode(PdePreprocessor.Mode mode)

write

public PreprocessorResult write(Writer out,
                                String program)
                         throws SketchException,
                                antlr.RecognitionException,
                                antlr.TokenStreamException

Throws:

SketchException

antlr.RecognitionException

antlr.TokenStreamException

write

public PreprocessorResult write(Writer out,
                                String program,
                                StringList codeFolderPackages)
                         throws SketchException,
                                antlr.RecognitionException,
                                antlr.TokenStreamException

Throws:

SketchException

antlr.RecognitionException

antlr.TokenStreamException

writeParseTree

protected void writeParseTree(String filename,
                              antlr.collections.AST ast)

writeImports

protected int writeImports(PrintWriter out,
                           List<String> programImports,
                           List<String> codeFolderImports)

Parameters:

out -

programImports -

codeFolderImports -

Returns:

the header offset

writeImportList

protected int writeImportList(PrintWriter out,
                              List<String> imports)

writeImportList

protected int writeImportList(PrintWriter out,
                              String[] imports)

writeDeclaration

protected void writeDeclaration(PrintWriter out,
                                String className)

Write any required header material (eg imports, class decl stuff)

Parameters:

out - PrintStream to write it to.

exporting - Is this being exported from PDE?

className - Name of the class being created.

writeFooter

protected void writeFooter(PrintWriter out,
                           String className)

Write any necessary closing text.

Parameters:

out - PrintStream to write it to.

getCoreImports

public String[] getCoreImports()

getDefaultImports

public String[] getDefaultImports()

ignoreImport

public boolean ignoreImport(String pkg)

Return true if this import should be removed from the code. This is used for packages like processing.xml which no longer exist.

Parameters:

pkg - something like processing.xml.XMLElement or processing.xml.*

Returns:

true if this shouldn't be added to the final code

debugAST

public void debugAST(antlr.collections.AST ast,
                     boolean includeHidden)