class Ruby::Prism::Translation::Ripper

This class provides a compatibility layer between prism and Ripper. It functions by parsing the entire tree first and then walking it and executing each of the Ripper callbacks as it goes. To use this class, you treat Ruby::Prism::Translation::Ripper effectively as you would treat the Ripper class.

Note that this class will serve the most common use cases, but Ripper’s API is extensive and undocumented. It relies on reporting the state of the parser at any given time. We do our best to replicate that here, but because it is a different architecture it is not possible to perfectly replicate the behavior of Ripper.

The main known difference is that we may omit dispatching some events in some cases. This impacts the following events:

on_assign_error
on_comma
on_ignored_nl
on_ignored_sp
on_nl
on_operator_ambiguous
on_semicolon
on_sp

Constants

EVENTS: This array contains name of all ripper events.
PARSER_EVENTS: This array contains name of parser events.
PARSER_EVENT_TABLE: This contains a table of all of the parser events and their corresponding arity.
SCANNER_EVENTS: This array contains name of scanner events.
SCANNER_EVENT_TABLE: This contains a table of all of the scanner events and their corresponding arity.

Attributes

column [R]

The current column in bytes of the parser.

filename [R]

The filename of the source being parsed.

lineno [R]

The current line number of the parser.

source [R]

The source that is being parsed.

Public Class Methods

lex (src, filename = "-", lineno = 1, raise_errors: false)

Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token, state]. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.lex("def m(a) nil end")
#=> [[[1,  0], :on_kw,     "def", FNAME    ],
     [[1,  3], :on_sp,     " ",   FNAME    ],
     [[1,  4], :on_ident,  "m",   ENDFN    ],
     [[1,  5], :on_lparen, "(",   BEG|LABEL],
     [[1,  6], :on_ident,  "a",   ARG      ],
     [[1,  7], :on_rparen, ")",   ENDFN    ],
     [[1,  8], :on_sp,     " ",   BEG      ],
     [[1,  9], :on_kw,     "nil", END      ],
     [[1, 12], :on_sp,     " ",   END      ],
     [[1, 13], :on_kw,     "end", END      ]]

# File lib/ruby/prism/translation/ripper.rb, line 59
def self.lex(src, filename = "-", lineno = 1, raise_errors: false)
  coerced = coerce_source(src)
  result = Ruby::Prism.lex_compat(coerced, filepath: filename, line: lineno, version: "current", encoding: coerced.encoding)

  if result.failure? && raise_errors
    raise SyntaxError, result.errors.first.message
  else
    result.value
  end
end

new (source, filename = "(ripper)", lineno = 1)

Create a new Translation::Ripper object with the given source.

# File lib/ruby/prism/translation/ripper.rb, line 544
def initialize(source, filename = "(ripper)", lineno = 1)
  @source = Ripper.coerce_source(source)
  @filename = filename
  @lineno = lineno
  @column = 0
  @result = nil
  @line_and_column_cache = nil
end

parse (src, filename = "(ripper)", lineno = 1)

Parses the given Ruby program read from src. src must be a String or an IO or a object with a gets method.

# File lib/ruby/prism/translation/ripper.rb, line 33
def self.parse(src, filename = "(ripper)", lineno = 1)
  new(src, filename, lineno).parse
end

sexp (src, filename = "-", lineno = 1, raise_errors: false)

Parses src and create S-exp tree. Returns more readable tree rather than Ripper.sexp_raw. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp("def m(a) nil end")
  #=> [:program,
       [[:def,
        [:@ident, "m", [1, 4]],
        [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil, nil, nil]],
        [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]

# File lib/ruby/prism/translation/ripper.rb, line 397
def self.sexp(src, filename = "-", lineno = 1, raise_errors: false)
  builder = SexpBuilderPP.new(src, filename, lineno)
  sexp = builder.parse
  if builder.error?
    if raise_errors
      raise SyntaxError, builder.error
    end
  else
    sexp
  end
end

sexp_raw (src, filename = "-", lineno = 1, raise_errors: false)

Parses src and create S-exp tree. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp_raw("def m(a) nil end")
  #=> [:program,
       [:stmts_add,
        [:stmts_new],
        [:def,
         [:@ident, "m", [1, 4]],
         [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]],
         [:bodystmt,
          [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]],
          nil,
          nil,
          nil]]]]

# File lib/ruby/prism/translation/ripper.rb, line 432
def self.sexp_raw(src, filename = "-", lineno = 1, raise_errors: false)
  builder = SexpBuilder.new(src, filename, lineno)
  sexp = builder.parse
  if builder.error?
    if raise_errors
      raise SyntaxError, builder.error
    end
  else
    sexp
  end
end

tokenize (...)

Tokenizes the Ruby program and returns an array of strings. The filename and lineno arguments are mostly ignored, since the return value is just the tokenized input. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.

p Ripper.tokenize(“def m(a) nil end”) # => [“def”, “ ”, “m”, “(”, “a”, “)”, “ ”, “nil”, “ ”, “end”]

# File lib/ruby/prism/translation/ripper.rb, line 79
def self.tokenize(...)
  lex(...).map { |token| token[2] }
end

Public Instance Methods

encoding ()

Return encoding of the source.

# File lib/ruby/prism/translation/ripper.rb, line 652
def encoding
  result.source.encoding
end

end_seen? ()

Return true if parsed source ended by __END__.

# File lib/ruby/prism/translation/ripper.rb, line 657
def end_seen?
  !!result.data_loc
end

error? ()

True if the parser encountered an error during parsing.

# File lib/ruby/prism/translation/ripper.rb, line 558
def error?
  result.failure?
end

parse ()

Parse the source and return the result.

# File lib/ruby/prism/translation/ripper.rb, line 563
def parse
  result.comments.each do |comment|
    location = comment.location
    bounds(location)

    if comment.is_a?(InlineComment)
      # Inline comments always contain a newline if the line itself contains it
      if result.source.source.bytesize > comment.location.end_offset
        on_comment("#{comment.slice}\n")
      else
        on_comment(comment.slice)
      end
    else
      offset = location.start_offset
      lines = comment.slice.lines

      lines.each_with_index do |line, index|
        bounds(location.copy(start_offset: offset))

        if index == 0
          on_embdoc_beg(line)
        elsif index == lines.size - 1
          on_embdoc_end(line)
        else
          on_embdoc(line)
        end

        offset += line.bytesize
      end
    end
  end

  result.magic_comments.each do |magic_comment|
    on_magic_comment(magic_comment.key, magic_comment.value)
  end

  unless result.data_loc.nil?
    on___end__(result.data_loc.slice.each_line.first)
  end

  result.warnings.each do |warning|
    bounds(warning.location)

    if warning.level == :default
      warning(warning.message)
    else
      case warning.type
      when :ambiguous_first_argument_plus
        on_arg_ambiguous("+")
      when :ambiguous_first_argument_minus
        on_arg_ambiguous("-")
      when :ambiguous_slash
        on_arg_ambiguous("/")
      else
        warn(warning.message)
      end
    end
  end

  if error?
    result.errors.each do |error|
      location = error.location
      bounds(location)

      case error.type
      when :alias_argument
        on_alias_error("can't make alias for the number variables", location.slice)
      when :argument_formal_class
        on_param_error("formal argument cannot be a class variable", location.slice)
      when :argument_format_constant
        on_param_error("formal argument cannot be a constant", location.slice)
      when :argument_formal_global
        on_param_error("formal argument cannot be a global variable", location.slice)
      when :argument_formal_ivar
        on_param_error("formal argument cannot be an instance variable", location.slice)
      when :class_name, :module_name
        on_class_name_error("class/module name must be CONSTANT", location.slice)
      else
        on_parse_error(error.message)
      end
    end

    nil
  else
    result.value.accept(self)
  end
end