Go

Concurrent and Systems Programming

grep
regular expressions, goroutines, pipelines

04/14/10 grep – 1
04/14/10 grep – 2
grep/grep.go
package grep; import ( "bufio"; "io"; "os"; "regexp"; "strings" )

func Grep(ptn string, writer io.Writer, reader io.Reader) os.Error {

if pat, ep := regexp.Compile(ptn); ep != nil { return ep } else {
out := bufio.NewWriter(writer); in := bufio.NewReader(reader)
loop: for {
switch line, er := in.ReadString('\n'); er {
case nil:
for _, body := range strings.Split(line, "\n", 0) {
if pat.MatchString(body) {
if _, ew := out.WriteString(body + "\n"); ew != nil {
return ew
} } }
case os.EOF: break loop
default: return er
}
}
if ef := out.Flush(); ef != nil { return ef }
}
return nil
}
04/14/10 grep – 3
 Reader  Writer
import "io"

type Writer interface {

Write(p []byte) (n int, err os.Error)
}
type Reader interface {
Read(p []byte) (n int, err os.Error)
}

type ReadWriter interface {

Reader // promotes the method
Writer
}

wrap essential method so that many types can

support the interface.
more, e.g., Closer, Seeker

04/14/10 grep – 4

http://golang.org/pkg/io/#Reader
http://golang.org/pkg/io/#Writer
 bufio
import "bufio"

func NewReader(rd io.Reader) *Reader

func (*Reader) ReadString // and many others

func NewWriter(wr io.Writer) *Writer

func (*Writer) WriteString // and many others

supports buffered and delimiter-oriented

reading and writing.

slightly byte-aware.

04/14/10 grep – 5

http://golang.org/pkg/bufio/
 regexp
import "regexp"

func MatchString(pat string, s string) (matched bool, err os.Error)

func Compile(pat string) (regexp *Regexp, err os.Error)
func (re *Regexp) MatchStrings(s string) (a []string)
func (re *Regexp) ReplaceAllString(src, repl string) string
// and many others

Unix- (not Perl-) like regular expressions.

Alternatives, character classes, and * + ?.
Subexpression matches with simple ( ).

04/14/10 grep – 6

http://golang.org/pkg/regexp/
grep/main.go
package main

import ( "flag"; "fmt"; "grep"; "os" )

func main() {
flag.Parse()
if flag.NArg() != 1 { // arguments?
fmt.Fprintln(os.Stderr, "usage: grep regular-expression")
os.Exit(1)
}
if err := grep.Grep(flag.Arg(0), os.Stdout, os.Stdin); err != nil {
fmt.Fprintf(os.Stderr, "grep error: %s\n", err)
os.Exit(1)
}
}

could allow files as per cat.

04/14/10 grep – 7
grep/mgrep.go   init
package main; import ( "flag"; "fmt"; "grep"; "io"; "os"; "runtime" )

func main() {
fflag := flag.Bool("f", false, "use file system pipe")
flag.Parse()
if flag.NArg() != 1 { // arguments?
fmt.Fprintln(os.Stderr, "usage: mgrep [-f] regular-expression")
os.Exit(1)
}
pipe := func() (io.Reader, io.WriteCloser, os.Error) { // -f?
if *fflag { return os.Pipe() }
i, o := io.Pipe(); return i, o, nil
}

os.Pipe uses a Unix pipe, i.e., inter-process.

io.Pipe is channel-based, i.e., intra-process.

04/14/10 grep – 8
grep/mgrep.go   one step
runtime.GOMAXPROCS(2) // arrange to schedule 2 CPUs

var err os.Error // accumulate errors

run := func(name string, o io.WriteCloser, i io.Reader) {
if eg := grep.Grep(name, o, i); eg != nil { err = eg } // one grep
if ec := o.Close(); ec != nil { err = ec } // close output!
}
}

any function call can be run concurrently.

a writer needs to close a pipe for a reader to
find end-of-file.
beware of multiple writers...

04/14/10 grep – 9
grep/mgrep.go   pipeline
var in io.Reader = os.Stdin // first input from stdin

for a := 0; a < flag.NArg()-1; a++ { // all but last as goroutines

if i, o, ep := os.Pipe(); ep == nil { // other i/o pipelined
go run(flag.Arg(a), o, in) // concurrently
in = i
} else { err = ep; break }
}

if err == nil { // run last directly

run(flag.Arg(flag.NArg()-1), os.Stdout, in) // output to stdout
}
if err != nil { // check (cumulative) errors
fmt.Fprintf(os.Stderr, "mgrep: %s\n", err)
os.Exit(1)
}
}

can report only a single error.

04/14/10 grep – 10
 go
go function-call-expression

go func() { ... } ()

go executes a function call concurrently with

the subsequent flow of execution.
go creates a (cheap) goroutine.
the function can be a closure; access to global
things must be monitored.
CPU scheduling is not (yet) automatic.

04/14/10 grep – 11

http://golang.org/doc/go_spec.html#Go_statements
 runtime

interacts with Go’s runtime system, e.g.,

infrastructure for reflection

goroutine management
memory management
...

04/14/10 grep – 12

http://golang.org/pkg/runtime/