class Ripper - ripper: Ruby Standard Library Documentation

# class Ripper

`Ripper` is a Ruby script parser.

You can get information from the parser with event-based style. Information such as abstract syntax trees or simple lexical analysis of the Ruby program.

## Usage¶ ↑

`Ripper` provides an easy interface for parsing your program into a symbolic expression tree (or S-expression).

Understanding the output of the parser may come as a challenge, it’s recommended you use PP to format the output for legibility.

    require 'ripper'
    require 'pp'

    pp Ripper.sexp('def hello(world) "Hello, #{world}!"; end')
      #=> [:program,
           [[:def,
             [:@ident, "hello", [1, 4]],
             [:paren,
              [:params, [[:@ident, "world", [1, 10]]], nil, nil, nil, nil, nil, nil]],
             [:bodystmt,
              [[:string_literal,
                [:string_content,
                 [:@tstring_content, "Hello, ", [1, 18]],
                 [:string_embexpr, [[:var_ref, [:@ident, "world", [1, 27]]]]],
                 [:@tstring_content, "!", [1, 33]]]]],
              nil,
              nil,
              nil]]]]

You can see in the example above, the expression starts with `:program`.

From here, a method definition at `:def`, followed by the method’s identifier `:@ident`. After the method’s identifier comes the parentheses `:paren` and the method parameters under `:params`.

Next is the method body, starting at `:bodystmt` (`stmt` meaning statement), which contains the full definition of the method.

In our case, we’re simply returning a String, so next we have the `:string_literal` expression.

Within our `:string_literal` you’ll notice two `@tstring_content`, this is the literal part for `Hello,` and `!`. Between the two `@tstring_content` statements is a `:string_embexpr`, where embexpr is an embedded expression. Our expression consists of a local variable, or `var_ref`, with the identifier (`@ident`) of `world`.

## Resources¶ ↑

- Ruby Inside

## Requirements¶ ↑

- ruby 1.9 (support CVS HEAD only)

- bison 1.28 or later (Other yaccs do not work)

## License¶ ↑

Ruby License.

- Minero Aoki

- aamine@loveruby.net

- i.loveruby.net

### Constants

EVENTS

  This array contains name of all ripper events.

EXPR_ARG

  newline significant, +/- is an operator.

EXPR_ARG_ANY

  equals to `(EXPR_ARG | EXPR_CMDARG)`

EXPR_BEG

  ignore newline, +/- is a sign.

EXPR_BEG_ANY

  equals to `(EXPR_BEG | EXPR_MID | EXPR_CLASS)`

EXPR_CLASS

  immediate after ‘class’, no here document.

EXPR_CMDARG

  newline significant, +/- is an operator.

EXPR_DOT

  ‘, no reserved words.

EXPR_END

  newline significant, +/- is an operator.

EXPR_ENDARG

  ditto, and unbound braces.

EXPR_ENDFN

  ditto, and unbound braces.

EXPR_END_ANY

  equals to `(EXPR_END | EXPR_ENDARG | EXPR_ENDFN)`

EXPR_FITEM

  symbol literal as FNAME.

EXPR_FNAME

  ignore newline, no reserved words.

EXPR_LABEL

  flag bit, label is allowed.

EXPR_LABELED

  flag bit, just after a label.

EXPR_MID

  newline significant, +/- is an operator.

EXPR_NONE

  equals to `0`

EXPR_VALUE

  equals to `EXPR_BEG`

PARSER_EVENTS

  This array contains name of parser events.

SCANNER_EVENTS

  This array contains name of scanner events.

Version

  version of `Ripper`

### Public Class Methods

dedent_string(p1, p2) click to toggle source

    static VALUE
    parser_dedent_string(VALUE self, VALUE input, VALUE width)
    {
        struct parser_params *p;
        struct dedent_string_arg args;

        p = rb_parser_params_new();

        args.p = p;
        args.input = input;
        args.width = width;
        return rb_ensure(parser_dedent_string0, (VALUE)&args, parser_free, (VALUE)p);
    }

lex(src, filename = '-', lineno = 1, **kw) click to toggle source

Tokenizes the Ruby program and returns an array of an array, which is formatted like `[[lineno, column], type, token, state]`. The `filename` argument is mostly ignored. By default, this method does not handle syntax errors in `src`, use the `raise_errors` keyword to raise a SyntaxError for an error in `src`.

    require 'ripper'
    require 'pp'

    pp Ripper.lex("def m(a) nil end")
    #=> [[[1,  0], :on_kw,     "def", FNAME    ],
         [[1,  3], :on_sp,     " ",   FNAME    ],
         [[1,  4], :on_ident,  "m",   ENDFN    ],
         [[1,  5], :on_lparen, "(",   BEG|LABEL],
         [[1,  6], :on_ident,  "a",   ARG      ],
         [[1,  7], :on_rparen, ")",   ENDFN    ],
         [[1,  8], :on_sp,     " ",   BEG      ],
         [[1,  9], :on_kw,     "nil", END      ],
         [[1, 12], :on_sp,     " ",   END      ],
         [[1, 13], :on_kw,     "end", END      ]]

    # File ripper/lib/ripper/lexer.rb, line 51
    def Ripper.lex(src, filename = '-', lineno = 1, **kw)
      Lexer.new(src, filename, lineno).lex(**kw)
    end

lex_state_name(p1) click to toggle source

    static VALUE
    ripper_lex_state_name(VALUE self, VALUE state)
    {
        struct parser_params *p;
        struct lex_state_name_arg args;

        p = rb_parser_params_new();

        args.p = p;
        args.state = state;

        return rb_ensure(lex_state_name0, (VALUE)&args, parser_free, (VALUE)p);
    }

new(src, filename="(ripper)", lineno=1) → ripper click to toggle source

Create a new `Ripper` object. src must be a String, an IO, or an `Object` which has gets method.

This method does not starts parsing. See also `Ripper#parse` and `Ripper.parse`.

    static VALUE
    ripper_initialize(int argc, VALUE *argv, VALUE self)
    {
        struct ripper *r;
        struct parser_params *p;
        VALUE src, fname, lineno;
        rb_parser_lex_gets_func *gets;
        VALUE sourcefile_string;
        const char *sourcefile;
        int sourceline;
        rb_parser_input_data input;

        p = ripper_parser_params(self, false);
        TypedData_Get_Struct(self, struct ripper, &parser_data_type, r);
        rb_scan_args(argc, argv, "12", &src, &fname, &lineno);
        if (RB_TYPE_P(src, T_FILE)) {
            gets = ripper_lex_io_get;
            r->type = lex_type_io;
            r->data.val = src;
            input = (rb_parser_input_data)src;
        }
        else if (rb_respond_to(src, id_gets)) {
            gets = ripper_lex_get_generic;
            r->type = lex_type_generic;
            r->data.val = src;
            input = (rb_parser_input_data)src;
        }
        else {
            StringValue(src);
            gets = ripper_lex_get_str;
            r->type = lex_type_str;
            r->data.ptr_str.str = src;
            r->data.ptr_str.ptr = 0;
            input = (rb_parser_input_data)&r->data.ptr_str;
        }
        if (NIL_P(fname)) {
            fname = STR_NEW2("(ripper)");
            OBJ_FREEZE(fname);
        }
        else {
            StringValueCStr(fname);
            fname = rb_str_new_frozen(fname);
        }
        rb_ruby_ripper_parser_initialize(p);

        sourcefile_string = fname;
        sourcefile = RSTRING_PTR(fname);
        sourceline = NIL_P(lineno) ? 0 : NUM2INT(lineno) - 1;

        rb_ruby_parser_ripper_initialize(p, gets, input, sourcefile_string, sourcefile, sourceline);

        return Qnil;
    }

parse(src, filename = '(ripper)', lineno = 1) click to toggle source

Parses the given Ruby program read from `src`. `src` must be a String or an IO or a object with a gets method.

    # File ripper/lib/ripper/core.rb, line 18
    def Ripper.parse(src, filename = '(ripper)', lineno = 1)
      new(src, filename, lineno).parse
    end

sexp(src, filename = '-', lineno = 1, raise_errors: false) click to toggle source

EXPERIMENTAL

  Parses `src` and create S-exp tree. Returns more readable tree rather than `Ripper.sexp_raw`. This method is mainly for developer use. The `filename` argument is mostly ignored. By default, this method does not handle syntax errors in `src`, returning `nil` in such cases. Use the `raise_errors` keyword to raise a SyntaxError for an error in `src`.

      require 'ripper'
      require 'pp'

      pp Ripper.sexp("def m(a) nil end")
        #=> [:program,
             [[:def,
              [:@ident, "m", [1, 4]],
              [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil, nil, nil]],
              [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]

    # File ripper/lib/ripper/sexp.rb, line 35
    def Ripper.sexp(src, filename = '-', lineno = 1, raise_errors: false)
      builder = SexpBuilderPP.new(src, filename, lineno)
      sexp = builder.parse
      if builder.error?
        if raise_errors
          raise SyntaxError, builder.error
        end
      else
        sexp
      end
    end

sexp_raw(src, filename = '-', lineno = 1, raise_errors: false) click to toggle source

EXPERIMENTAL

  Parses `src` and create S-exp tree. This method is mainly for developer use. The `filename` argument is mostly ignored. By default, this method does not handle syntax errors in `src`, returning `nil` in such cases. Use the `raise_errors` keyword to raise a SyntaxError for an error in `src`.

      require 'ripper'
      require 'pp'

      pp Ripper.sexp_raw("def m(a) nil end")
        #=> [:program,
             [:stmts_add,
              [:stmts_new],
              [:def,
               [:@ident, "m", [1, 4]],
               [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]],
               [:bodystmt,
                [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]],
                nil,
                nil,
                nil]]]]

    # File ripper/lib/ripper/sexp.rb, line 71
    def Ripper.sexp_raw(src, filename = '-', lineno = 1, raise_errors: false)
      builder = SexpBuilder.new(src, filename, lineno)
      sexp = builder.parse
      if builder.error?
        if raise_errors
          raise SyntaxError, builder.error
        end
      else
        sexp
      end
    end

slice(src, pattern, n = 0) click to toggle source

EXPERIMENTAL

  Parses `src` and return a string which was matched to `pattern`. `pattern` should be described as Regexp.

      require 'ripper'

      p Ripper.slice('def m(a) nil end', 'ident')                   #=> "m"
      p Ripper.slice('def m(a) nil end', '[ident lparen rparen]+')  #=> "m(a)"
      p Ripper.slice("<<EOS\nstring\nEOS",
                     'heredoc_beg nl $(tstring_content*) heredoc_end', 1)
          #=> "string\n"

    # File ripper/lib/ripper/lexer.rb, line 277
    def Ripper.slice(src, pattern, n = 0)
      if m = token_match(src, pattern)
      then m.string(n)
      else nil
      end
    end

tokenize(src, filename = '-', lineno = 1, **kw) click to toggle source

Tokenizes the Ruby program and returns an array of strings. The `filename` and `lineno` arguments are mostly ignored, since the return value is just the tokenized input. By default, this method does not handle syntax errors in `src`, use the `raise_errors` keyword to raise a SyntaxError for an error in `src`.

    p Ripper.tokenize("def m(a) nil end")
       # => ["def", " ", "m", "(", "a", ")", " ", "nil", " ", "end"]

    # File ripper/lib/ripper/lexer.rb, line 25
    def Ripper.tokenize(src, filename = '-', lineno = 1, **kw)
      Lexer.new(src, filename, lineno).tokenize(**kw)
    end

### Public Instance Methods

column → Integer click to toggle source

Return column number of current parsing line. This number starts from 0.

    static VALUE
    ripper_column(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);
        long col;

        if (NIL_P(rb_ruby_parser_parsing_thread(p))) return Qnil;
        col = rb_ruby_ripper_column(p);
        return LONG2NUM(col);
    }

debug_output → obj click to toggle source

Get debug output.

    static VALUE
    ripper_parser_get_debug_output(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, false);

        return rb_ruby_parser_debug_output(p);
    }

debug_output = obj click to toggle source

Set debug output.

    static VALUE
    ripper_parser_set_debug_output(VALUE self, VALUE output)
    {
        struct parser_params *p = ripper_parser_params(self, false);

        rb_ruby_parser_set_debug_output(p, output);
        return output;
    }

encoding → encoding click to toggle source

Return encoding of the source.

    static VALUE
    ripper_parser_encoding(VALUE vparser)
    {
        struct parser_params *p = ripper_parser_params(vparser, false);

        return rb_enc_from_encoding(rb_ruby_parser_encoding(p));
    }

end_seen? → Boolean click to toggle source

Return true if parsed source ended by +_END_+.

    static VALUE
    ripper_parser_end_seen_p(VALUE vparser)
    {
        struct parser_params *p = ripper_parser_params(vparser, false);

        return RBOOL(rb_ruby_parser_end_seen_p(p));
    }

error? → Boolean click to toggle source

Return true if parsed source has errors.

    static VALUE
    ripper_error_p(VALUE vparser)
    {
        struct parser_params *p = ripper_parser_params(vparser, false);

        return RBOOL(rb_ruby_parser_error_p(p));
    }

filename → String click to toggle source

Return current parsing filename.

    static VALUE
    ripper_filename(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);

        return rb_ruby_parser_ruby_sourcefile_string(p);
    }

lineno → Integer click to toggle source

Return line number of current parsing line. This number starts from 1.

    static VALUE
    ripper_lineno(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);

        if (NIL_P(rb_ruby_parser_parsing_thread(p))) return Qnil;
        return INT2NUM(rb_ruby_parser_ruby_sourceline(p));
    }

parse click to toggle source

Start parsing and returns the value of the root action.

    static VALUE
    ripper_parse(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);
        VALUE result;

        if (!NIL_P(rb_ruby_parser_parsing_thread(p))) {
            if (rb_ruby_parser_parsing_thread(p) == rb_thread_current())
                rb_raise(rb_eArgError, "Ripper#parse is not reentrant");
            else
                rb_raise(rb_eArgError, "Ripper#parse is not multithread-safe");
        }
        rb_ruby_parser_set_parsing_thread(p, rb_thread_current());
        result = rb_ensure(ripper_parse0, self, ripper_ensure, self);
        RB_GC_GUARD(self);

        return result;
    }

state → Integer click to toggle source

Return scanner state of current token.

    static VALUE
    ripper_state(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);

        if (NIL_P(rb_ruby_parser_parsing_thread(p))) return Qnil;
        return INT2NUM(rb_ruby_parser_lex_state(p));
    }

token → String click to toggle source

Return the current token string.

    static VALUE
    ripper_token(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, true);
        long pos, len;
        VALUE str;

        if (NIL_P(rb_ruby_parser_parsing_thread(p))) return Qnil;
        pos = rb_ruby_ripper_column(p);
        len = rb_ruby_ripper_token_len(p);
        str = rb_str_new_parser_string(rb_ruby_ripper_lex_lastline(p));
        return rb_str_subseq(str, pos, len);
    }

yydebug → true or false click to toggle source

Get yydebug.

    static VALUE
    ripper_parser_get_yydebug(VALUE self)
    {
        struct parser_params *p = ripper_parser_params(self, false);

        return RBOOL(rb_ruby_parser_get_yydebug(p));
    }

yydebug = flag click to toggle source

Set yydebug.

    static VALUE
    ripper_parser_set_yydebug(VALUE self, VALUE flag)
    {
        struct parser_params *p = ripper_parser_params(self, false);

        rb_ruby_parser_set_yydebug(p, RTEST(flag));
        return flag;
    }

### Private Instance Methods

compile_error(msg) click to toggle source

This method is called when the parser found syntax error.

    # File ripper/lib/ripper/core.rb, line 63
    def compile_error(msg)
    end

dedent_string(p1, p2) click to toggle source

    static VALUE
    parser_dedent_string(VALUE self, VALUE input, VALUE width)
    {
        struct parser_params *p;
        struct dedent_string_arg args;

        p = rb_parser_params_new();

        args.p = p;
        args.input = input;
        args.width = width;
        return rb_ensure(parser_dedent_string0, (VALUE)&args, parser_free, (VALUE)p);
    }

Also aliased as: dedent_string
warn(fmt, *args) click to toggle source

This method is called when weak warning is produced by the parser. `fmt` and `args` is printf style.

    # File ripper/lib/ripper/core.rb, line 54
    def warn(fmt, *args)
    end

warning(fmt, *args) click to toggle source

This method is called when strong warning is produced by the parser. `fmt` and `args` is printf style.

    # File ripper/lib/ripper/core.rb, line 59
    def warning(fmt, *args)
    end