Commit eb4d90f1 authored by Victor Marmol's avatar Victor Marmol

Update go-dockerclient godep.

Adds support for new Docker remote API.
parent 2c81050e
...@@ -173,7 +173,8 @@ ...@@ -173,7 +173,8 @@
}, },
{ {
"ImportPath": "github.com/fsouza/go-dockerclient", "ImportPath": "github.com/fsouza/go-dockerclient",
"Rev": "09334c56c63bab2cd6c4ccab924d89e2419a361f" "Comment": "0.2.1-532-g2f1ad24",
"Rev": "2f1ad24900b2777139b5becee93eb63a75b00617"
}, },
{ {
"ImportPath": "github.com/garyburd/redigo/internal", "ImportPath": "github.com/garyburd/redigo/internal",
......
...@@ -6,7 +6,5 @@ go: ...@@ -6,7 +6,5 @@ go:
env: env:
- GOARCH=amd64 - GOARCH=amd64
- GOARCH=386 - GOARCH=386
install:
- make testdeps
script: script:
- make test - make test
...@@ -21,6 +21,7 @@ David Huie <dahuie@gmail.com> ...@@ -21,6 +21,7 @@ David Huie <dahuie@gmail.com>
Dawn Chen <dawnchen@google.com> Dawn Chen <dawnchen@google.com>
Ed <edrocksit@gmail.com> Ed <edrocksit@gmail.com>
Eric Anderson <anderson@copperegg.com> Eric Anderson <anderson@copperegg.com>
Ewout Prangsma <ewout@prangsma.net>
Fabio Rehm <fgrehm@gmail.com> Fabio Rehm <fgrehm@gmail.com>
Fatih Arslan <ftharsln@gmail.com> Fatih Arslan <ftharsln@gmail.com>
Flavia Missi <flaviamissi@gmail.com> Flavia Missi <flaviamissi@gmail.com>
...@@ -37,6 +38,7 @@ Johan Euphrosine <proppy@google.com> ...@@ -37,6 +38,7 @@ Johan Euphrosine <proppy@google.com>
Kamil Domanski <kamil@domanski.co> Kamil Domanski <kamil@domanski.co>
Karan Misra <kidoman@gmail.com> Karan Misra <kidoman@gmail.com>
Kim, Hirokuni <hirokuni.kim@kvh.co.jp> Kim, Hirokuni <hirokuni.kim@kvh.co.jp>
Kyle Allan <kallan357@gmail.com>
liron-l <levinlir@gmail.com> liron-l <levinlir@gmail.com>
Lucas Clemente <lucas@clemente.io> Lucas Clemente <lucas@clemente.io>
Lucas Weiblen <lucasweiblen@gmail.com> Lucas Weiblen <lucasweiblen@gmail.com>
...@@ -67,6 +69,7 @@ Summer Mousa <smousa@zenoss.com> ...@@ -67,6 +69,7 @@ Summer Mousa <smousa@zenoss.com>
Tarsis Azevedo <tarsis@corp.globo.com> Tarsis Azevedo <tarsis@corp.globo.com>
Tim Schindler <tim@catalyst-zero.com> Tim Schindler <tim@catalyst-zero.com>
Tobi Knaup <tobi@mesosphere.io> Tobi Knaup <tobi@mesosphere.io>
Victor Marmol <vmarmol@google.com>
Vincenzo Prignano <vincenzo.prignano@gmail.com> Vincenzo Prignano <vincenzo.prignano@gmail.com>
Wiliam Souza <wiliamsouza83@gmail.com> Wiliam Souza <wiliamsouza83@gmail.com>
Ye Yin <eyniy@qq.com> Ye Yin <eyniy@qq.com>
......
...@@ -2,5 +2,5 @@ ...@@ -2,5 +2,5 @@
Version 2.0, January 2004 Version 2.0, January 2004
http://www.apache.org/licenses/ http://www.apache.org/licenses/
You can find the Docker license int the following link: You can find the Docker license at the following link:
https://raw2.github.com/dotcloud/docker/master/LICENSE https://raw.githubusercontent.com/docker/docker/master/LICENSE
.PHONY: \ .PHONY: \
all \ all \
deps \ vendor \
updatedeps \ lint \
testdeps \ vet \
updatetestdeps \ fmt \
cov \ fmtcheck \
pretest \
test \ test \
cov \
clean clean
all: test all: test
deps: vendor:
go get -d -v ./... go get -v github.com/mjibson/party
party -d vendor -c -u
lint:
go get -v github.com/golang/lint/golint
for file in $(shell git ls-files '*.go' | grep -v '^vendor/'); do \
golint $$file; \
done
updatedeps: vet:
go get -d -v -u -f ./... go get -v golang.org/x/tools/cmd/vet
go vet ./...
testdeps: fmt:
go get -d -v -t ./... gofmt -w $(shell git ls-files '*.go' | grep -v '^vendor/')
updatetestdeps: fmtcheck:
go get -d -v -t -u -f ./... for file in $(shell git ls-files '*.go' | grep -v '^vendor/'); do \
gofmt $$file | diff -u $$file -; \
if [ -n "$$(gofmt $$file | diff -u $$file -)" ]; then\
exit 1; \
fi; \
done
cov: testdeps pretest: lint vet fmtcheck
test: pretest
go test
go test ./testing
cov:
go get -v github.com/axw/gocov/gocov go get -v github.com/axw/gocov/gocov
go get golang.org/x/tools/cmd/cover go get golang.org/x/tools/cmd/cover
gocov test | gocov report gocov test | gocov report
test: testdeps
go test ./...
./testing/bin/fmtpolice
clean: clean:
go clean ./... go clean ./...
#go-dockerclient # go-dockerclient
[![Build Status](https://drone.io/github.com/fsouza/go-dockerclient/status.png)](https://drone.io/github.com/fsouza/go-dockerclient/latest) [![Drone](https://drone.io/github.com/fsouza/go-dockerclient/status.png)](https://drone.io/github.com/fsouza/go-dockerclient/latest)
[![Build Status](https://travis-ci.org/fsouza/go-dockerclient.png)](https://travis-ci.org/fsouza/go-dockerclient) [![Travis](https://img.shields.io/travis/fsouza/go-dockerclient.svg?style=flat-square)](https://travis-ci.org/fsouza/go-dockerclient)
[![GoDoc](https://img.shields.io/badge/api-Godoc-blue.svg?style=flat-square)](https://godoc.org/github.com/fsouza/go-dockerclient)
[![GoDoc](https://godoc.org/github.com/fsouza/go-dockerclient?status.png)](https://godoc.org/github.com/fsouza/go-dockerclient)
This package presents a client for the Docker remote API. It also provides This package presents a client for the Docker remote API. It also provides
support for the extensions in the [Swarm API](https://docs.docker.com/swarm/API/). support for the extensions in the [Swarm API](https://docs.docker.com/swarm/API/).
...@@ -64,6 +63,13 @@ func main() { ...@@ -64,6 +63,13 @@ func main() {
## Developing ## Developing
You can run the tests with: All development commands can be seen in the [Makefile](Makefile).
Commited code must pass:
* [golint](https://github.com/golang/lint)
* [go vet](https://godoc.org/golang.org/x/tools/cmd/vet)
* [gofmt](https://golang.org/cmd/gofmt)
* [go test](https://golang.org/cmd/go/#hdr-Test_packages)
make test Running `make test` will check all of these. If your editor does not automatically call gofmt, `make fmt` will format all go files in this repository.
...@@ -9,7 +9,7 @@ import ( ...@@ -9,7 +9,7 @@ import (
"reflect" "reflect"
"testing" "testing"
"github.com/docker/docker/pkg/archive" "github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/archive"
) )
func TestBuildImageMultipleContextsError(t *testing.T) { func TestBuildImageMultipleContextsError(t *testing.T) {
......
...@@ -23,6 +23,8 @@ import ( ...@@ -23,6 +23,8 @@ import (
"reflect" "reflect"
"strconv" "strconv"
"strings" "strings"
"github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/stdcopy"
) )
const userAgent = "go-dockerclient" const userAgent = "go-dockerclient"
...@@ -438,7 +440,7 @@ func (c *Client) stream(method, path string, streamOptions streamOptions) error ...@@ -438,7 +440,7 @@ func (c *Client) stream(method, path string, streamOptions streamOptions) error
if streamOptions.setRawTerminal { if streamOptions.setRawTerminal {
_, err = io.Copy(streamOptions.stdout, resp.Body) _, err = io.Copy(streamOptions.stdout, resp.Body)
} else { } else {
_, err = stdCopy(streamOptions.stdout, streamOptions.stderr, resp.Body) _, err = stdcopy.StdCopy(streamOptions.stdout, streamOptions.stderr, resp.Body)
} }
return err return err
} }
...@@ -509,31 +511,37 @@ func (c *Client) hijack(method, path string, hijackOptions hijackOptions) error ...@@ -509,31 +511,37 @@ func (c *Client) hijack(method, path string, hijackOptions hijackOptions) error
} }
rwc, br := clientconn.Hijack() rwc, br := clientconn.Hijack()
defer rwc.Close() defer rwc.Close()
errs := make(chan error, 2) errChanOut := make(chan error, 1)
errChanIn := make(chan error, 1)
exit := make(chan bool) exit := make(chan bool)
go func() { go func() {
defer close(exit) defer close(exit)
defer close(errChanOut)
var err error var err error
if hijackOptions.setRawTerminal { if hijackOptions.setRawTerminal {
// When TTY is ON, use regular copy // When TTY is ON, use regular copy
_, err = io.Copy(hijackOptions.stdout, br) _, err = io.Copy(hijackOptions.stdout, br)
} else { } else {
_, err = stdCopy(hijackOptions.stdout, hijackOptions.stderr, br) _, err = stdcopy.StdCopy(hijackOptions.stdout, hijackOptions.stderr, br)
} }
errs <- err errChanOut <- err
}() }()
go func() { go func() {
var err error
if hijackOptions.in != nil { if hijackOptions.in != nil {
_, err = io.Copy(rwc, hijackOptions.in) _, err := io.Copy(rwc, hijackOptions.in)
errChanIn <- err
} }
rwc.(interface { rwc.(interface {
CloseWrite() error CloseWrite() error
}).CloseWrite() }).CloseWrite()
errs <- err
}() }()
<-exit <-exit
return <-errs select {
case err = <-errChanIn:
return err
case err = <-errChanOut:
return err
}
} }
func (c *Client) getURL(path string) string { func (c *Client) getURL(path string) string {
......
...@@ -251,7 +251,9 @@ func TestGetList(t *testing.T) { ...@@ -251,7 +251,9 @@ func TestGetList(t *testing.T) {
func TestSetList(t *testing.T) { func TestSetList(t *testing.T) {
list := []string{"a", "b", "c"} list := []string{"a", "b", "c"}
var env Env var env Env
env.SetList("SOME", list) if err := env.SetList("SOME", list); err != nil {
t.Error(err)
}
if got := env.GetList("SOME"); !reflect.DeepEqual(got, list) { if got := env.GetList("SOME"); !reflect.DeepEqual(got, list) {
t.Errorf("Env.SetList(%v): wrong result. Got %v", list, got) t.Errorf("Env.SetList(%v): wrong result. Got %v", list, got)
} }
......
...@@ -57,7 +57,6 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han ...@@ -57,7 +57,6 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han
{"status":"destroy","id":"dfdf82bd3881","from":"base:latest","time":1374067970} {"status":"destroy","id":"dfdf82bd3881","from":"base:latest","time":1374067970}
` `
var req http.Request
server := buildServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { server := buildServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rsc := bufio.NewScanner(strings.NewReader(response)) rsc := bufio.NewScanner(strings.NewReader(response))
for rsc.Scan() { for rsc.Scan() {
...@@ -65,7 +64,6 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han ...@@ -65,7 +64,6 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han
w.(http.Flusher).Flush() w.(http.Flusher).Flush()
time.Sleep(10 * time.Millisecond) time.Sleep(10 * time.Millisecond)
} }
req = *r
})) }))
defer server.Close() defer server.Close()
...@@ -76,7 +74,12 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han ...@@ -76,7 +74,12 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han
client.SkipServerVersionCheck = true client.SkipServerVersionCheck = true
listener := make(chan *APIEvents, 10) listener := make(chan *APIEvents, 10)
defer func() { time.Sleep(10 * time.Millisecond); client.RemoveEventListener(listener) }() defer func() {
time.Sleep(10 * time.Millisecond)
if err := client.RemoveEventListener(listener); err != nil {
t.Error(err)
}
}()
err = client.AddEventListener(listener) err = client.AddEventListener(listener)
if err != nil { if err != nil {
...@@ -89,7 +92,7 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han ...@@ -89,7 +92,7 @@ func testEventListeners(testName string, t *testing.T, buildServer func(http.Han
for { for {
select { select {
case msg := <-listener: case msg := <-listener:
t.Logf("Received: %s", *msg) t.Logf("Received: %v", *msg)
count++ count++
err = checkEvent(count, msg) err = checkEvent(count, msg)
if err != nil { if err != nil {
......
...@@ -87,11 +87,9 @@ func TestExecStartDetached(t *testing.T) { ...@@ -87,11 +87,9 @@ func TestExecStartDetached(t *testing.T) {
func TestExecStartAndAttach(t *testing.T) { func TestExecStartAndAttach(t *testing.T) {
var reader = strings.NewReader("send value") var reader = strings.NewReader("send value")
var req http.Request
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte{1, 0, 0, 0, 0, 0, 0, 5}) w.Write([]byte{1, 0, 0, 0, 0, 0, 0, 5})
w.Write([]byte("hello")) w.Write([]byte("hello"))
req = *r
})) }))
defer server.Close() defer server.Close()
client, _ := NewClient(server.URL) client, _ := NewClient(server.URL)
...@@ -106,7 +104,11 @@ func TestExecStartAndAttach(t *testing.T) { ...@@ -106,7 +104,11 @@ func TestExecStartAndAttach(t *testing.T) {
RawTerminal: true, RawTerminal: true,
Success: success, Success: success,
} }
go client.StartExec(execID, opts) go func() {
if err := client.StartExec(execID, opts); err != nil {
t.Error(err)
}
}()
<-success <-success
} }
......
...@@ -104,7 +104,6 @@ var ( ...@@ -104,7 +104,6 @@ var (
// //
// See http://goo.gl/HRVN1Z for more details. // See http://goo.gl/HRVN1Z for more details.
func (c *Client) ListImages(opts ListImagesOptions) ([]APIImages, error) { func (c *Client) ListImages(opts ListImagesOptions) ([]APIImages, error) {
// TODO(pedge): what happens if we specify the digest parameter when using API Version <1.18?
path := "/images/json?" + queryString(opts) path := "/images/json?" + queryString(opts)
body, _, err := c.do("GET", path, doOptions{}) body, _, err := c.do("GET", path, doOptions{})
if err != nil { if err != nil {
...@@ -240,13 +239,17 @@ func (c *Client) PushImage(opts PushImageOptions, auth AuthConfiguration) error ...@@ -240,13 +239,17 @@ func (c *Client) PushImage(opts PushImageOptions, auth AuthConfiguration) error
if opts.Name == "" { if opts.Name == "" {
return ErrNoSuchImage return ErrNoSuchImage
} }
headers, err := headersWithAuth(auth)
if err != nil {
return err
}
name := opts.Name name := opts.Name
opts.Name = "" opts.Name = ""
path := "/images/" + name + "/push?" + queryString(&opts) path := "/images/" + name + "/push?" + queryString(&opts)
return c.stream("POST", path, streamOptions{ return c.stream("POST", path, streamOptions{
setRawTerminal: true, setRawTerminal: true,
rawJSONStream: opts.RawJSONStream, rawJSONStream: opts.RawJSONStream,
headers: headersWithAuth(auth), headers: headers,
stdout: opts.OutputStream, stdout: opts.OutputStream,
}) })
} }
...@@ -271,7 +274,10 @@ func (c *Client) PullImage(opts PullImageOptions, auth AuthConfiguration) error ...@@ -271,7 +274,10 @@ func (c *Client) PullImage(opts PullImageOptions, auth AuthConfiguration) error
return ErrNoSuchImage return ErrNoSuchImage
} }
headers := headersWithAuth(auth) headers, err := headersWithAuth(auth)
if err != nil {
return err
}
return c.createImage(queryString(&opts), headers, nil, opts.OutputStream, opts.RawJSONStream) return c.createImage(queryString(&opts), headers, nil, opts.OutputStream, opts.RawJSONStream)
} }
...@@ -387,8 +393,13 @@ type BuildImageOptions struct { ...@@ -387,8 +393,13 @@ type BuildImageOptions struct {
Dockerfile string `qs:"dockerfile"` Dockerfile string `qs:"dockerfile"`
NoCache bool `qs:"nocache"` NoCache bool `qs:"nocache"`
SuppressOutput bool `qs:"q"` SuppressOutput bool `qs:"q"`
Pull bool `qs:"pull"`
RmTmpContainer bool `qs:"rm"` RmTmpContainer bool `qs:"rm"`
ForceRmTmpContainer bool `qs:"forcerm"` ForceRmTmpContainer bool `qs:"forcerm"`
Memory int64 `qs:"memory"`
Memswap int64 `qs:"memswap"`
CPUShares int64 `qs:"cpushares"`
CPUSetCPUs string `qs:"cpusetcpus"`
InputStream io.Reader `qs:"-"` InputStream io.Reader `qs:"-"`
OutputStream io.Writer `qs:"-"` OutputStream io.Writer `qs:"-"`
RawJSONStream bool `qs:"-"` RawJSONStream bool `qs:"-"`
...@@ -401,12 +412,15 @@ type BuildImageOptions struct { ...@@ -401,12 +412,15 @@ type BuildImageOptions struct {
// BuildImage builds an image from a tarball's url or a Dockerfile in the input // BuildImage builds an image from a tarball's url or a Dockerfile in the input
// stream. // stream.
// //
// See http://goo.gl/wRsW76 for more details. // See http://goo.gl/7nuGXa for more details.
func (c *Client) BuildImage(opts BuildImageOptions) error { func (c *Client) BuildImage(opts BuildImageOptions) error {
if opts.OutputStream == nil { if opts.OutputStream == nil {
return ErrMissingOutputStream return ErrMissingOutputStream
} }
var headers = headersWithAuth(opts.Auth, opts.AuthConfigs) headers, err := headersWithAuth(opts.Auth, opts.AuthConfigs)
if err != nil {
return err
}
if opts.Remote != "" && opts.Name == "" { if opts.Remote != "" && opts.Name == "" {
opts.Name = opts.Remote opts.Name = opts.Remote
...@@ -421,7 +435,7 @@ func (c *Client) BuildImage(opts BuildImageOptions) error { ...@@ -421,7 +435,7 @@ func (c *Client) BuildImage(opts BuildImageOptions) error {
return ErrMultipleContexts return ErrMultipleContexts
} }
var err error var err error
if opts.InputStream, err = createTarStream(opts.ContextDir); err != nil { if opts.InputStream, err = createTarStream(opts.ContextDir, opts.Dockerfile); err != nil {
return err return err
} }
} }
...@@ -469,23 +483,27 @@ func isURL(u string) bool { ...@@ -469,23 +483,27 @@ func isURL(u string) bool {
return p.Scheme == "http" || p.Scheme == "https" return p.Scheme == "http" || p.Scheme == "https"
} }
func headersWithAuth(auths ...interface{}) map[string]string { func headersWithAuth(auths ...interface{}) (map[string]string, error) {
var headers = make(map[string]string) var headers = make(map[string]string)
for _, auth := range auths { for _, auth := range auths {
switch auth.(type) { switch auth.(type) {
case AuthConfiguration: case AuthConfiguration:
var buf bytes.Buffer var buf bytes.Buffer
json.NewEncoder(&buf).Encode(auth) if err := json.NewEncoder(&buf).Encode(auth); err != nil {
return nil, err
}
headers["X-Registry-Auth"] = base64.URLEncoding.EncodeToString(buf.Bytes()) headers["X-Registry-Auth"] = base64.URLEncoding.EncodeToString(buf.Bytes())
case AuthConfigurations: case AuthConfigurations:
var buf bytes.Buffer var buf bytes.Buffer
json.NewEncoder(&buf).Encode(auth) if err := json.NewEncoder(&buf).Encode(auth); err != nil {
return nil, err
}
headers["X-Registry-Config"] = base64.URLEncoding.EncodeToString(buf.Bytes()) headers["X-Registry-Config"] = base64.URLEncoding.EncodeToString(buf.Bytes())
} }
} }
return headers return headers, nil
} }
// APIImageSearch reflect the result of a search on the dockerHub // APIImageSearch reflect the result of a search on the dockerHub
......
...@@ -240,7 +240,9 @@ func TestInspectImage(t *testing.T) { ...@@ -240,7 +240,9 @@ func TestInspectImage(t *testing.T) {
"container_config":{"Memory":0} "container_config":{"Memory":0}
}` }`
var expected Image var expected Image
json.Unmarshal([]byte(body), &expected) if err := json.Unmarshal([]byte(body), &expected); err != nil {
t.Fatal(err)
}
fakeRT := &FakeRoundTripper{message: body, status: http.StatusOK} fakeRT := &FakeRoundTripper{message: body, status: http.StatusOK}
client := newTestClient(fakeRT) client := newTestClient(fakeRT)
image, err := client.InspectImage(expected.ID) image, err := client.InspectImage(expected.ID)
...@@ -655,8 +657,13 @@ func TestBuildImageParameters(t *testing.T) { ...@@ -655,8 +657,13 @@ func TestBuildImageParameters(t *testing.T) {
Name: "testImage", Name: "testImage",
NoCache: true, NoCache: true,
SuppressOutput: true, SuppressOutput: true,
Pull: true,
RmTmpContainer: true, RmTmpContainer: true,
ForceRmTmpContainer: true, ForceRmTmpContainer: true,
Memory: 1024,
Memswap: 2048,
CPUShares: 10,
CPUSetCPUs: "0-3",
InputStream: &buf, InputStream: &buf,
OutputStream: &buf, OutputStream: &buf,
} }
...@@ -665,7 +672,18 @@ func TestBuildImageParameters(t *testing.T) { ...@@ -665,7 +672,18 @@ func TestBuildImageParameters(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
req := fakeRT.requests[0] req := fakeRT.requests[0]
expected := map[string][]string{"t": {opts.Name}, "nocache": {"1"}, "q": {"1"}, "rm": {"1"}, "forcerm": {"1"}} expected := map[string][]string{
"t": {opts.Name},
"nocache": {"1"},
"q": {"1"},
"pull": {"1"},
"rm": {"1"},
"forcerm": {"1"},
"memory": {"1024"},
"memswap": {"2048"},
"cpushares": {"10"},
"cpusetcpus": {"0-3"},
}
got := map[string][]string(req.URL.Query()) got := map[string][]string(req.URL.Query())
if !reflect.DeepEqual(got, expected) { if !reflect.DeepEqual(got, expected) {
t.Errorf("BuildImage: wrong query string. Want %#v. Got %#v.", expected, got) t.Errorf("BuildImage: wrong query string. Want %#v. Got %#v.", expected, got)
......
// Copyright 2014 Docker authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the DOCKER-LICENSE file.
package docker
import (
"encoding/binary"
"errors"
"io"
)
const (
stdWriterPrefixLen = 8
stdWriterFdIndex = 0
stdWriterSizeIndex = 4
)
var errInvalidStdHeader = errors.New("Unrecognized input header")
func stdCopy(dstout, dsterr io.Writer, src io.Reader) (written int64, err error) {
var (
buf = make([]byte, 32*1024+stdWriterPrefixLen+1)
bufLen = len(buf)
nr, nw int
er, ew error
out io.Writer
frameSize int
)
for {
for nr < stdWriterPrefixLen {
var nr2 int
nr2, er = src.Read(buf[nr:])
if er == io.EOF {
if nr < stdWriterPrefixLen && nr2 < stdWriterPrefixLen {
return written, nil
}
nr += nr2
break
} else if er != nil {
return 0, er
}
nr += nr2
}
switch buf[stdWriterFdIndex] {
case 0:
fallthrough
case 1:
out = dstout
case 2:
out = dsterr
default:
return 0, errInvalidStdHeader
}
frameSize = int(binary.BigEndian.Uint32(buf[stdWriterSizeIndex : stdWriterSizeIndex+4]))
if frameSize+stdWriterPrefixLen > bufLen {
buf = append(buf, make([]byte, frameSize+stdWriterPrefixLen-len(buf)+1)...)
bufLen = len(buf)
}
for nr < frameSize+stdWriterPrefixLen {
var nr2 int
nr2, er = src.Read(buf[nr:])
if er == io.EOF {
if nr == 0 {
return written, nil
}
nr += nr2
break
} else if er != nil {
return 0, er
}
nr += nr2
}
bound := frameSize + stdWriterPrefixLen
if bound > nr {
bound = nr
}
nw, ew = out.Write(buf[stdWriterPrefixLen:bound])
if nw > 0 {
written += int64(nw)
}
if ew != nil {
return 0, ew
}
if nw != frameSize {
return written, io.ErrShortWrite
}
copy(buf, buf[frameSize+stdWriterPrefixLen:])
nr -= frameSize + stdWriterPrefixLen
}
}
// Copyright 2014 go-dockerclient authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the DOCKER-LICENSE file.
package docker
import (
"bytes"
"encoding/binary"
"errors"
"io"
"strings"
"testing"
"testing/iotest"
)
type errorWriter struct {
}
func (errorWriter) Write([]byte) (int, error) {
return 0, errors.New("something went wrong")
}
func TestStdCopy(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
input.Write([]byte("something happened!"))
input.Write([]byte{1, 0, 0, 0, 0, 0, 0, 12})
input.Write([]byte("just kidding"))
input.Write([]byte{0, 0, 0, 0, 0, 0, 0, 6})
input.Write([]byte("\nyeah!"))
n, err := stdCopy(&stdout, &stderr, &input)
if err != nil {
t.Fatal(err)
}
if expected := int64(19 + 12 + 6); n != expected {
t.Errorf("Wrong number of bytes. Want %d. Got %d.", expected, n)
}
if got := stderr.String(); got != "something happened!" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "something happened!", got)
}
if got := stdout.String(); got != "just kidding\nyeah!" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "just kidding\nyeah!", got)
}
}
func TestStdCopyStress(t *testing.T) {
var input, stdout, stderr bytes.Buffer
value := strings.Repeat("something ", 4096)
writer := newStdWriter(&input, Stdout)
writer.Write([]byte(value))
n, err := stdCopy(&stdout, &stderr, &input)
if err != nil {
t.Fatal(err)
}
if n != 40960 {
t.Errorf("Wrong number of bytes. Want 40960. Got %d.", n)
}
if got := stderr.String(); got != "" {
t.Errorf("stdCopy: wrong stderr. Want empty string. Got %q", got)
}
if got := stdout.String(); got != value {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q", value, got)
}
}
func TestStdCopyInvalidStdHeader(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{3, 0, 0, 0, 0, 0, 0, 19})
n, err := stdCopy(&stdout, &stderr, &input)
if n != 0 {
t.Errorf("stdCopy: wrong number of bytes. Want 0. Got %d", n)
}
if err != errInvalidStdHeader {
t.Errorf("stdCopy: wrong error. Want ErrInvalidStdHeader. Got %#v", err)
}
}
func TestStdCopyBigFrame(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 18})
input.Write([]byte("something happened!"))
n, err := stdCopy(&stdout, &stderr, &input)
if err != nil {
t.Fatal(err)
}
if expected := int64(18); n != expected {
t.Errorf("Wrong number of bytes. Want %d. Got %d.", expected, n)
}
if got := stderr.String(); got != "something happened" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "something happened", got)
}
if got := stdout.String(); got != "" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "", got)
}
}
func TestStdCopySmallFrame(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 20})
input.Write([]byte("something happened!"))
n, err := stdCopy(&stdout, &stderr, &input)
if err != io.ErrShortWrite {
t.Errorf("stdCopy: wrong error. Want ShortWrite. Got %#v", err)
}
if expected := int64(19); n != expected {
t.Errorf("Wrong number of bytes. Want %d. Got %d.", expected, n)
}
if got := stderr.String(); got != "something happened!" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "something happened", got)
}
if got := stdout.String(); got != "" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "", got)
}
}
func TestStdCopyEmpty(t *testing.T) {
var input, stdout, stderr bytes.Buffer
n, err := stdCopy(&stdout, &stderr, &input)
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("stdCopy: wrong number of bytes. Want 0. Got %d.", n)
}
}
func TestStdCopyCorruptedHeader(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0})
n, err := stdCopy(&stdout, &stderr, &input)
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("stdCopy: wrong number of bytes. Want 0. Got %d.", n)
}
}
func TestStdCopyTruncateWriter(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
input.Write([]byte("something happened!"))
n, err := stdCopy(&stdout, iotest.TruncateWriter(&stderr, 7), &input)
if err != nil {
t.Fatal(err)
}
if expected := int64(19); n != expected {
t.Errorf("Wrong number of bytes. Want %d. Got %d.", expected, n)
}
if got := stderr.String(); got != "somethi" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "somethi", got)
}
if got := stdout.String(); got != "" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "", got)
}
}
func TestStdCopyHeaderOnly(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
n, err := stdCopy(&stdout, iotest.TruncateWriter(&stderr, 7), &input)
if err != io.ErrShortWrite {
t.Errorf("stdCopy: wrong error. Want ShortWrite. Got %#v", err)
}
if n != 0 {
t.Errorf("Wrong number of bytes. Want 0. Got %d.", n)
}
if got := stderr.String(); got != "" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "", got)
}
if got := stdout.String(); got != "" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "", got)
}
}
func TestStdCopyDataErrReader(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
input.Write([]byte("something happened!"))
n, err := stdCopy(&stdout, &stderr, iotest.DataErrReader(&input))
if err != nil {
t.Fatal(err)
}
if expected := int64(19); n != expected {
t.Errorf("Wrong number of bytes. Want %d. Got %d.", expected, n)
}
if got := stderr.String(); got != "something happened!" {
t.Errorf("stdCopy: wrong stderr. Want %q. Got %q.", "something happened!", got)
}
if got := stdout.String(); got != "" {
t.Errorf("stdCopy: wrong stdout. Want %q. Got %q.", "", got)
}
}
func TestStdCopyTimeoutReader(t *testing.T) {
var input, stdout, stderr bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
input.Write([]byte("something happened!"))
_, err := stdCopy(&stdout, &stderr, iotest.TimeoutReader(&input))
if err != iotest.ErrTimeout {
t.Errorf("stdCopy: wrong error. Want ErrTimeout. Got %#v.", err)
}
}
func TestStdCopyWriteError(t *testing.T) {
var input bytes.Buffer
input.Write([]byte{2, 0, 0, 0, 0, 0, 0, 19})
input.Write([]byte("something happened!"))
var stdout, stderr errorWriter
n, err := stdCopy(stdout, stderr, &input)
if err.Error() != "something went wrong" {
t.Errorf("stdCopy: wrong error. Want %q. Got %q", "something went wrong", err)
}
if n != 0 {
t.Errorf("stdCopy: wrong number of bytes. Want 0. Got %d.", n)
}
}
type StdType [8]byte
var (
Stdin = StdType{0: 0}
Stdout = StdType{0: 1}
Stderr = StdType{0: 2}
)
type StdWriter struct {
io.Writer
prefix StdType
sizeBuf []byte
}
func (w *StdWriter) Write(buf []byte) (n int, err error) {
if w == nil || w.Writer == nil {
return 0, errors.New("Writer not instanciated")
}
binary.BigEndian.PutUint32(w.prefix[4:], uint32(len(buf)))
buf = append(w.prefix[:], buf...)
n, err = w.Writer.Write(buf)
return n - 8, err
}
func newStdWriter(w io.Writer, t StdType) *StdWriter {
if len(t) != 8 {
return nil
}
return &StdWriter{
Writer: w,
prefix: t,
sizeBuf: make([]byte, 4),
}
}
...@@ -13,21 +13,48 @@ import ( ...@@ -13,21 +13,48 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/docker/docker/pkg/archive" "github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/fileutils" "github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/fileutils"
) )
func createTarStream(srcPath string) (io.ReadCloser, error) { func createTarStream(srcPath, dockerfilePath string) (io.ReadCloser, error) {
excludes, err := parseDockerignore(srcPath) excludes, err := parseDockerignore(srcPath)
if err != nil { if err != nil {
return nil, err return nil, err
} }
includes := []string{"."}
// If .dockerignore mentions .dockerignore or the Dockerfile
// then make sure we send both files over to the daemon
// because Dockerfile is, obviously, needed no matter what, and
// .dockerignore is needed to know if either one needs to be
// removed. The deamon will remove them for us, if needed, after it
// parses the Dockerfile.
//
// https://github.com/docker/docker/issues/8330
//
forceIncludeFiles := []string{".dockerignore", dockerfilePath}
for _, includeFile := range forceIncludeFiles {
if includeFile == "" {
continue
}
keepThem, err := fileutils.Matches(includeFile, excludes)
if err != nil {
return nil, fmt.Errorf("cannot match .dockerfile: '%s', error: %s", includeFile, err)
}
if keepThem {
includes = append(includes, includeFile)
}
}
if err := validateContextDirectory(srcPath, excludes); err != nil { if err := validateContextDirectory(srcPath, excludes); err != nil {
return nil, err return nil, err
} }
tarOpts := &archive.TarOptions{ tarOpts := &archive.TarOptions{
ExcludePatterns: excludes, ExcludePatterns: excludes,
IncludeFiles: includes,
Compression: archive.Uncompressed, Compression: archive.Uncompressed,
NoLchown: true, NoLchown: true,
} }
...@@ -84,16 +111,7 @@ func parseDockerignore(root string) ([]string, error) { ...@@ -84,16 +111,7 @@ func parseDockerignore(root string) ([]string, error) {
if err != nil && !os.IsNotExist(err) { if err != nil && !os.IsNotExist(err) {
return excludes, fmt.Errorf("error reading .dockerignore: '%s'", err) return excludes, fmt.Errorf("error reading .dockerignore: '%s'", err)
} }
for _, pattern := range strings.Split(string(ignore), "\n") { excludes = strings.Split(string(ignore), "\n")
matches, err := filepath.Match(pattern, "Dockerfile")
if err != nil {
return excludes, fmt.Errorf("bad .dockerignore pattern: '%s', error: %s", pattern, err)
}
if matches {
return excludes, fmt.Errorf("dockerfile was excluded by .dockerignore pattern '%s'", pattern)
}
excludes = append(excludes, pattern)
}
return excludes, nil return excludes, nil
} }
...@@ -9,7 +9,7 @@ main() { ...@@ -9,7 +9,7 @@ main() {
check_fmt() { check_fmt() {
eval "set -e" eval "set -e"
for file in $(git ls-files '*.go') ; do for file in $(_list_go_files) ; do
gofmt $file | diff -u $file - gofmt $file | diff -u $file -
done done
eval "set +e" eval "set +e"
...@@ -18,7 +18,7 @@ check_fmt() { ...@@ -18,7 +18,7 @@ check_fmt() {
check_lint() { check_lint() {
_install_linter _install_linter
for file in $(git ls-files '*.go') ; do for file in $(_list_go_files) ; do
if [[ ! "$(${GOPATH}/bin/golint $file)" =~ ^[[:blank:]]*$ ]] ; then if [[ ! "$(${GOPATH}/bin/golint $file)" =~ ^[[:blank:]]*$ ]] ; then
_lint_verbose && exit 1 _lint_verbose && exit 1
fi fi
...@@ -26,7 +26,7 @@ check_lint() { ...@@ -26,7 +26,7 @@ check_lint() {
} }
_lint_verbose() { _lint_verbose() {
for file in $(git ls-files '*.go') ; do $GOPATH/bin/golint $file ; done for file in $(_list_go_files) ; do $GOPATH/bin/golint $file ; done
} }
_install_linter() { _install_linter() {
...@@ -35,4 +35,8 @@ _install_linter() { ...@@ -35,4 +35,8 @@ _install_linter() {
fi fi
} }
_list_go_files() {
git ls-files '*.go' | grep -v '^vendor/'
}
main "$@" main "$@"
...@@ -22,7 +22,7 @@ import ( ...@@ -22,7 +22,7 @@ import (
"time" "time"
"github.com/fsouza/go-dockerclient" "github.com/fsouza/go-dockerclient"
"github.com/gorilla/mux" "github.com/fsouza/go-dockerclient/vendor/github.com/gorilla/mux"
) )
var nameRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`) var nameRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`)
...@@ -533,8 +533,19 @@ func (s *DockerServer) attachContainer(w http.ResponseWriter, r *http.Request) { ...@@ -533,8 +533,19 @@ func (s *DockerServer) attachContainer(w http.ResponseWriter, r *http.Request) {
http.Error(w, err.Error(), http.StatusNotFound) http.Error(w, err.Error(), http.StatusNotFound)
return return
} }
outStream := newStdWriter(w, stdout) hijacker, ok := w.(http.Hijacker)
fmt.Fprintf(outStream, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n") if !ok {
http.Error(w, "cannot hijack connection", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/vnd.docker.raw-stream")
w.WriteHeader(http.StatusOK)
conn, _, err := hijacker.Hijack()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
outStream := newStdWriter(conn, stdout)
if container.State.Running { if container.State.Running {
fmt.Fprintf(outStream, "Container %q is running\n", container.ID) fmt.Fprintf(outStream, "Container %q is running\n", container.ID)
} else { } else {
...@@ -542,6 +553,7 @@ func (s *DockerServer) attachContainer(w http.ResponseWriter, r *http.Request) { ...@@ -542,6 +553,7 @@ func (s *DockerServer) attachContainer(w http.ResponseWriter, r *http.Request) {
} }
fmt.Fprintln(outStream, "What happened?") fmt.Fprintln(outStream, "What happened?")
fmt.Fprintln(outStream, "Something happened") fmt.Fprintln(outStream, "Something happened")
conn.Close()
} }
func (s *DockerServer) waitContainer(w http.ResponseWriter, r *http.Request) { func (s *DockerServer) waitContainer(w http.ResponseWriter, r *http.Request) {
...@@ -566,12 +578,13 @@ func (s *DockerServer) waitContainer(w http.ResponseWriter, r *http.Request) { ...@@ -566,12 +578,13 @@ func (s *DockerServer) waitContainer(w http.ResponseWriter, r *http.Request) {
func (s *DockerServer) removeContainer(w http.ResponseWriter, r *http.Request) { func (s *DockerServer) removeContainer(w http.ResponseWriter, r *http.Request) {
id := mux.Vars(r)["id"] id := mux.Vars(r)["id"]
force := r.URL.Query().Get("force")
_, index, err := s.findContainer(id) _, index, err := s.findContainer(id)
if err != nil { if err != nil {
http.Error(w, err.Error(), http.StatusNotFound) http.Error(w, err.Error(), http.StatusNotFound)
return return
} }
if s.containers[index].State.Running { if s.containers[index].State.Running && force != "1" {
msg := "Error: API error (406): Impossible to remove a running container, please stop it first" msg := "Error: API error (406): Impossible to remove a running container, please stop it first"
http.Error(w, msg, http.StatusInternalServerError) http.Error(w, msg, http.StatusInternalServerError)
return return
......
...@@ -850,6 +850,23 @@ func TestRemoveContainerRunning(t *testing.T) { ...@@ -850,6 +850,23 @@ func TestRemoveContainerRunning(t *testing.T) {
} }
} }
func TestRemoveContainerRunningForce(t *testing.T) {
server := DockerServer{}
addContainers(&server, 1)
server.containers[0].State.Running = true
server.buildMuxer()
recorder := httptest.NewRecorder()
path := fmt.Sprintf("/containers/%s?%s", server.containers[0].ID, "force=1")
request, _ := http.NewRequest("DELETE", path, nil)
server.ServeHTTP(recorder, request)
if recorder.Code != http.StatusNoContent {
t.Errorf("RemoveContainer: wrong status. Want %d. Got %d.", http.StatusNoContent, recorder.Code)
}
if len(server.containers) > 0 {
t.Error("RemoveContainer: did not remove the container.")
}
}
func TestPullImage(t *testing.T) { func TestPullImage(t *testing.T) {
server := DockerServer{imgIDs: make(map[string]string)} server := DockerServer{imgIDs: make(map[string]string)}
server.buildMuxer() server.buildMuxer()
......
# 0.8
logrus: defaults to stderr instead of stdout
# 0.7.3
formatter/\*: allow configuration of timestamp layout
# 0.7.2
formatter/text: Add configuration option for time format (#158)
The MIT License (MIT)
Copyright (c) 2014 Simon Eskildsen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
package logrus
import (
"bytes"
"fmt"
"io"
"os"
"time"
)
// An entry is the final or intermediate Logrus logging entry. It contains all
// the fields passed with WithField{,s}. It's finally logged when Debug, Info,
// Warn, Error, Fatal or Panic is called on it. These objects can be reused and
// passed around as much as you wish to avoid field duplication.
type Entry struct {
Logger *Logger
// Contains all the fields set by the user.
Data Fields
// Time at which the log entry was created
Time time.Time
// Level the log entry was logged at: Debug, Info, Warn, Error, Fatal or Panic
Level Level
// Message passed to Debug, Info, Warn, Error, Fatal or Panic
Message string
}
func NewEntry(logger *Logger) *Entry {
return &Entry{
Logger: logger,
// Default is three fields, give a little extra room
Data: make(Fields, 5),
}
}
// Returns a reader for the entry, which is a proxy to the formatter.
func (entry *Entry) Reader() (*bytes.Buffer, error) {
serialized, err := entry.Logger.Formatter.Format(entry)
return bytes.NewBuffer(serialized), err
}
// Returns the string representation from the reader and ultimately the
// formatter.
func (entry *Entry) String() (string, error) {
reader, err := entry.Reader()
if err != nil {
return "", err
}
return reader.String(), err
}
// Add a single field to the Entry.
func (entry *Entry) WithField(key string, value interface{}) *Entry {
return entry.WithFields(Fields{key: value})
}
// Add a map of fields to the Entry.
func (entry *Entry) WithFields(fields Fields) *Entry {
data := Fields{}
for k, v := range entry.Data {
data[k] = v
}
for k, v := range fields {
data[k] = v
}
return &Entry{Logger: entry.Logger, Data: data}
}
func (entry *Entry) log(level Level, msg string) {
entry.Time = time.Now()
entry.Level = level
entry.Message = msg
if err := entry.Logger.Hooks.Fire(level, entry); err != nil {
entry.Logger.mu.Lock()
fmt.Fprintf(os.Stderr, "Failed to fire hook: %v\n", err)
entry.Logger.mu.Unlock()
}
reader, err := entry.Reader()
if err != nil {
entry.Logger.mu.Lock()
fmt.Fprintf(os.Stderr, "Failed to obtain reader, %v\n", err)
entry.Logger.mu.Unlock()
}
entry.Logger.mu.Lock()
defer entry.Logger.mu.Unlock()
_, err = io.Copy(entry.Logger.Out, reader)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to write to log, %v\n", err)
}
// To avoid Entry#log() returning a value that only would make sense for
// panic() to use in Entry#Panic(), we avoid the allocation by checking
// directly here.
if level <= PanicLevel {
panic(entry)
}
}
func (entry *Entry) Debug(args ...interface{}) {
if entry.Logger.Level >= DebugLevel {
entry.log(DebugLevel, fmt.Sprint(args...))
}
}
func (entry *Entry) Print(args ...interface{}) {
entry.Info(args...)
}
func (entry *Entry) Info(args ...interface{}) {
if entry.Logger.Level >= InfoLevel {
entry.log(InfoLevel, fmt.Sprint(args...))
}
}
func (entry *Entry) Warn(args ...interface{}) {
if entry.Logger.Level >= WarnLevel {
entry.log(WarnLevel, fmt.Sprint(args...))
}
}
func (entry *Entry) Warning(args ...interface{}) {
entry.Warn(args...)
}
func (entry *Entry) Error(args ...interface{}) {
if entry.Logger.Level >= ErrorLevel {
entry.log(ErrorLevel, fmt.Sprint(args...))
}
}
func (entry *Entry) Fatal(args ...interface{}) {
if entry.Logger.Level >= FatalLevel {
entry.log(FatalLevel, fmt.Sprint(args...))
}
os.Exit(1)
}
func (entry *Entry) Panic(args ...interface{}) {
if entry.Logger.Level >= PanicLevel {
entry.log(PanicLevel, fmt.Sprint(args...))
}
panic(fmt.Sprint(args...))
}
// Entry Printf family functions
func (entry *Entry) Debugf(format string, args ...interface{}) {
if entry.Logger.Level >= DebugLevel {
entry.Debug(fmt.Sprintf(format, args...))
}
}
func (entry *Entry) Infof(format string, args ...interface{}) {
if entry.Logger.Level >= InfoLevel {
entry.Info(fmt.Sprintf(format, args...))
}
}
func (entry *Entry) Printf(format string, args ...interface{}) {
entry.Infof(format, args...)
}
func (entry *Entry) Warnf(format string, args ...interface{}) {
if entry.Logger.Level >= WarnLevel {
entry.Warn(fmt.Sprintf(format, args...))
}
}
func (entry *Entry) Warningf(format string, args ...interface{}) {
entry.Warnf(format, args...)
}
func (entry *Entry) Errorf(format string, args ...interface{}) {
if entry.Logger.Level >= ErrorLevel {
entry.Error(fmt.Sprintf(format, args...))
}
}
func (entry *Entry) Fatalf(format string, args ...interface{}) {
if entry.Logger.Level >= FatalLevel {
entry.Fatal(fmt.Sprintf(format, args...))
}
}
func (entry *Entry) Panicf(format string, args ...interface{}) {
if entry.Logger.Level >= PanicLevel {
entry.Panic(fmt.Sprintf(format, args...))
}
}
// Entry Println family functions
func (entry *Entry) Debugln(args ...interface{}) {
if entry.Logger.Level >= DebugLevel {
entry.Debug(entry.sprintlnn(args...))
}
}
func (entry *Entry) Infoln(args ...interface{}) {
if entry.Logger.Level >= InfoLevel {
entry.Info(entry.sprintlnn(args...))
}
}
func (entry *Entry) Println(args ...interface{}) {
entry.Infoln(args...)
}
func (entry *Entry) Warnln(args ...interface{}) {
if entry.Logger.Level >= WarnLevel {
entry.Warn(entry.sprintlnn(args...))
}
}
func (entry *Entry) Warningln(args ...interface{}) {
entry.Warnln(args...)
}
func (entry *Entry) Errorln(args ...interface{}) {
if entry.Logger.Level >= ErrorLevel {
entry.Error(entry.sprintlnn(args...))
}
}
func (entry *Entry) Fatalln(args ...interface{}) {
if entry.Logger.Level >= FatalLevel {
entry.Fatal(entry.sprintlnn(args...))
}
}
func (entry *Entry) Panicln(args ...interface{}) {
if entry.Logger.Level >= PanicLevel {
entry.Panic(entry.sprintlnn(args...))
}
}
// Sprintlnn => Sprint no newline. This is to get the behavior of how
// fmt.Sprintln where spaces are always added between operands, regardless of
// their type. Instead of vendoring the Sprintln implementation to spare a
// string allocation, we do the simplest thing.
func (entry *Entry) sprintlnn(args ...interface{}) string {
msg := fmt.Sprintln(args...)
return msg[:len(msg)-1]
}
package logrus
import (
"bytes"
"fmt"
"testing"
"github.com/fsouza/go-dockerclient/vendor/github.com/stretchr/testify/assert"
)
func TestEntryPanicln(t *testing.T) {
errBoom := fmt.Errorf("boom time")
defer func() {
p := recover()
assert.NotNil(t, p)
switch pVal := p.(type) {
case *Entry:
assert.Equal(t, "kaboom", pVal.Message)
assert.Equal(t, errBoom, pVal.Data["err"])
default:
t.Fatalf("want type *Entry, got %T: %#v", pVal, pVal)
}
}()
logger := New()
logger.Out = &bytes.Buffer{}
entry := NewEntry(logger)
entry.WithField("err", errBoom).Panicln("kaboom")
}
func TestEntryPanicf(t *testing.T) {
errBoom := fmt.Errorf("boom again")
defer func() {
p := recover()
assert.NotNil(t, p)
switch pVal := p.(type) {
case *Entry:
assert.Equal(t, "kaboom true", pVal.Message)
assert.Equal(t, errBoom, pVal.Data["err"])
default:
t.Fatalf("want type *Entry, got %T: %#v", pVal, pVal)
}
}()
logger := New()
logger.Out = &bytes.Buffer{}
entry := NewEntry(logger)
entry.WithField("err", errBoom).Panicf("kaboom %v", true)
}
package logrus
import (
"io"
)
var (
// std is the name of the standard logger in stdlib `log`
std = New()
)
func StandardLogger() *Logger {
return std
}
// SetOutput sets the standard logger output.
func SetOutput(out io.Writer) {
std.mu.Lock()
defer std.mu.Unlock()
std.Out = out
}
// SetFormatter sets the standard logger formatter.
func SetFormatter(formatter Formatter) {
std.mu.Lock()
defer std.mu.Unlock()
std.Formatter = formatter
}
// SetLevel sets the standard logger level.
func SetLevel(level Level) {
std.mu.Lock()
defer std.mu.Unlock()
std.Level = level
}
// GetLevel returns the standard logger level.
func GetLevel() Level {
std.mu.Lock()
defer std.mu.Unlock()
return std.Level
}
// AddHook adds a hook to the standard logger hooks.
func AddHook(hook Hook) {
std.mu.Lock()
defer std.mu.Unlock()
std.Hooks.Add(hook)
}
// WithField creates an entry from the standard logger and adds a field to
// it. If you want multiple fields, use `WithFields`.
//
// Note that it doesn't log until you call Debug, Print, Info, Warn, Fatal
// or Panic on the Entry it returns.
func WithField(key string, value interface{}) *Entry {
return std.WithField(key, value)
}
// WithFields creates an entry from the standard logger and adds multiple
// fields to it. This is simply a helper for `WithField`, invoking it
// once for each field.
//
// Note that it doesn't log until you call Debug, Print, Info, Warn, Fatal
// or Panic on the Entry it returns.
func WithFields(fields Fields) *Entry {
return std.WithFields(fields)
}
// Debug logs a message at level Debug on the standard logger.
func Debug(args ...interface{}) {
std.Debug(args...)
}
// Print logs a message at level Info on the standard logger.
func Print(args ...interface{}) {
std.Print(args...)
}
// Info logs a message at level Info on the standard logger.
func Info(args ...interface{}) {
std.Info(args...)
}
// Warn logs a message at level Warn on the standard logger.
func Warn(args ...interface{}) {
std.Warn(args...)
}
// Warning logs a message at level Warn on the standard logger.
func Warning(args ...interface{}) {
std.Warning(args...)
}
// Error logs a message at level Error on the standard logger.
func Error(args ...interface{}) {
std.Error(args...)
}
// Panic logs a message at level Panic on the standard logger.
func Panic(args ...interface{}) {
std.Panic(args...)
}
// Fatal logs a message at level Fatal on the standard logger.
func Fatal(args ...interface{}) {
std.Fatal(args...)
}
// Debugf logs a message at level Debug on the standard logger.
func Debugf(format string, args ...interface{}) {
std.Debugf(format, args...)
}
// Printf logs a message at level Info on the standard logger.
func Printf(format string, args ...interface{}) {
std.Printf(format, args...)
}
// Infof logs a message at level Info on the standard logger.
func Infof(format string, args ...interface{}) {
std.Infof(format, args...)
}
// Warnf logs a message at level Warn on the standard logger.
func Warnf(format string, args ...interface{}) {
std.Warnf(format, args...)
}
// Warningf logs a message at level Warn on the standard logger.
func Warningf(format string, args ...interface{}) {
std.Warningf(format, args...)
}
// Errorf logs a message at level Error on the standard logger.
func Errorf(format string, args ...interface{}) {
std.Errorf(format, args...)
}
// Panicf logs a message at level Panic on the standard logger.
func Panicf(format string, args ...interface{}) {
std.Panicf(format, args...)
}
// Fatalf logs a message at level Fatal on the standard logger.
func Fatalf(format string, args ...interface{}) {
std.Fatalf(format, args...)
}
// Debugln logs a message at level Debug on the standard logger.
func Debugln(args ...interface{}) {
std.Debugln(args...)
}
// Println logs a message at level Info on the standard logger.
func Println(args ...interface{}) {
std.Println(args...)
}
// Infoln logs a message at level Info on the standard logger.
func Infoln(args ...interface{}) {
std.Infoln(args...)
}
// Warnln logs a message at level Warn on the standard logger.
func Warnln(args ...interface{}) {
std.Warnln(args...)
}
// Warningln logs a message at level Warn on the standard logger.
func Warningln(args ...interface{}) {
std.Warningln(args...)
}
// Errorln logs a message at level Error on the standard logger.
func Errorln(args ...interface{}) {
std.Errorln(args...)
}
// Panicln logs a message at level Panic on the standard logger.
func Panicln(args ...interface{}) {
std.Panicln(args...)
}
// Fatalln logs a message at level Fatal on the standard logger.
func Fatalln(args ...interface{}) {
std.Fatalln(args...)
}
package logrus
import "time"
const DefaultTimestampFormat = time.RFC3339
// The Formatter interface is used to implement a custom Formatter. It takes an
// `Entry`. It exposes all the fields, including the default ones:
//
// * `entry.Data["msg"]`. The message passed from Info, Warn, Error ..
// * `entry.Data["time"]`. The timestamp.
// * `entry.Data["level"]. The level the entry was logged at.
//
// Any additional fields added with `WithField` or `WithFields` are also in
// `entry.Data`. Format is expected to return an array of bytes which are then
// logged to `logger.Out`.
type Formatter interface {
Format(*Entry) ([]byte, error)
}
// This is to not silently overwrite `time`, `msg` and `level` fields when
// dumping it. If this code wasn't there doing:
//
// logrus.WithField("level", 1).Info("hello")
//
// Would just silently drop the user provided level. Instead with this code
// it'll logged as:
//
// {"level": "info", "fields.level": 1, "msg": "hello", "time": "..."}
//
// It's not exported because it's still using Data in an opinionated way. It's to
// avoid code duplication between the two default formatters.
func prefixFieldClashes(data Fields) {
_, ok := data["time"]
if ok {
data["fields.time"] = data["time"]
}
_, ok = data["msg"]
if ok {
data["fields.msg"] = data["msg"]
}
_, ok = data["level"]
if ok {
data["fields.level"] = data["level"]
}
}
package logrus
import (
"testing"
"time"
)
// smallFields is a small size data set for benchmarking
var smallFields = Fields{
"foo": "bar",
"baz": "qux",
"one": "two",
"three": "four",
}
// largeFields is a large size data set for benchmarking
var largeFields = Fields{
"foo": "bar",
"baz": "qux",
"one": "two",
"three": "four",
"five": "six",
"seven": "eight",
"nine": "ten",
"eleven": "twelve",
"thirteen": "fourteen",
"fifteen": "sixteen",
"seventeen": "eighteen",
"nineteen": "twenty",
"a": "b",
"c": "d",
"e": "f",
"g": "h",
"i": "j",
"k": "l",
"m": "n",
"o": "p",
"q": "r",
"s": "t",
"u": "v",
"w": "x",
"y": "z",
"this": "will",
"make": "thirty",
"entries": "yeah",
}
func BenchmarkSmallTextFormatter(b *testing.B) {
doBenchmark(b, &TextFormatter{DisableColors: true}, smallFields)
}
func BenchmarkLargeTextFormatter(b *testing.B) {
doBenchmark(b, &TextFormatter{DisableColors: true}, largeFields)
}
func BenchmarkSmallColoredTextFormatter(b *testing.B) {
doBenchmark(b, &TextFormatter{ForceColors: true}, smallFields)
}
func BenchmarkLargeColoredTextFormatter(b *testing.B) {
doBenchmark(b, &TextFormatter{ForceColors: true}, largeFields)
}
func BenchmarkSmallJSONFormatter(b *testing.B) {
doBenchmark(b, &JSONFormatter{}, smallFields)
}
func BenchmarkLargeJSONFormatter(b *testing.B) {
doBenchmark(b, &JSONFormatter{}, largeFields)
}
func doBenchmark(b *testing.B, formatter Formatter, fields Fields) {
entry := &Entry{
Time: time.Time{},
Level: InfoLevel,
Message: "message",
Data: fields,
}
var d []byte
var err error
for i := 0; i < b.N; i++ {
d, err = formatter.Format(entry)
if err != nil {
b.Fatal(err)
}
b.SetBytes(int64(len(d)))
}
}
package logrus
import (
"testing"
"github.com/fsouza/go-dockerclient/vendor/github.com/stretchr/testify/assert"
)
type TestHook struct {
Fired bool
}
func (hook *TestHook) Fire(entry *Entry) error {
hook.Fired = true
return nil
}
func (hook *TestHook) Levels() []Level {
return []Level{
DebugLevel,
InfoLevel,
WarnLevel,
ErrorLevel,
FatalLevel,
PanicLevel,
}
}
func TestHookFires(t *testing.T) {
hook := new(TestHook)
LogAndAssertJSON(t, func(log *Logger) {
log.Hooks.Add(hook)
assert.Equal(t, hook.Fired, false)
log.Print("test")
}, func(fields Fields) {
assert.Equal(t, hook.Fired, true)
})
}
type ModifyHook struct {
}
func (hook *ModifyHook) Fire(entry *Entry) error {
entry.Data["wow"] = "whale"
return nil
}
func (hook *ModifyHook) Levels() []Level {
return []Level{
DebugLevel,
InfoLevel,
WarnLevel,
ErrorLevel,
FatalLevel,
PanicLevel,
}
}
func TestHookCanModifyEntry(t *testing.T) {
hook := new(ModifyHook)
LogAndAssertJSON(t, func(log *Logger) {
log.Hooks.Add(hook)
log.WithField("wow", "elephant").Print("test")
}, func(fields Fields) {
assert.Equal(t, fields["wow"], "whale")
})
}
func TestCanFireMultipleHooks(t *testing.T) {
hook1 := new(ModifyHook)
hook2 := new(TestHook)
LogAndAssertJSON(t, func(log *Logger) {
log.Hooks.Add(hook1)
log.Hooks.Add(hook2)
log.WithField("wow", "elephant").Print("test")
}, func(fields Fields) {
assert.Equal(t, fields["wow"], "whale")
assert.Equal(t, hook2.Fired, true)
})
}
type ErrorHook struct {
Fired bool
}
func (hook *ErrorHook) Fire(entry *Entry) error {
hook.Fired = true
return nil
}
func (hook *ErrorHook) Levels() []Level {
return []Level{
ErrorLevel,
}
}
func TestErrorHookShouldntFireOnInfo(t *testing.T) {
hook := new(ErrorHook)
LogAndAssertJSON(t, func(log *Logger) {
log.Hooks.Add(hook)
log.Info("test")
}, func(fields Fields) {
assert.Equal(t, hook.Fired, false)
})
}
func TestErrorHookShouldFireOnError(t *testing.T) {
hook := new(ErrorHook)
LogAndAssertJSON(t, func(log *Logger) {
log.Hooks.Add(hook)
log.Error("test")
}, func(fields Fields) {
assert.Equal(t, hook.Fired, true)
})
}
package logrus
// A hook to be fired when logging on the logging levels returned from
// `Levels()` on your implementation of the interface. Note that this is not
// fired in a goroutine or a channel with workers, you should handle such
// functionality yourself if your call is non-blocking and you don't wish for
// the logging calls for levels returned from `Levels()` to block.
type Hook interface {
Levels() []Level
Fire(*Entry) error
}
// Internal type for storing the hooks on a logger instance.
type levelHooks map[Level][]Hook
// Add a hook to an instance of logger. This is called with
// `log.Hooks.Add(new(MyHook))` where `MyHook` implements the `Hook` interface.
func (hooks levelHooks) Add(hook Hook) {
for _, level := range hook.Levels() {
hooks[level] = append(hooks[level], hook)
}
}
// Fire all the hooks for the passed level. Used by `entry.log` to fire
// appropriate hooks for a log entry.
func (hooks levelHooks) Fire(level Level, entry *Entry) error {
for _, hook := range hooks[level] {
if err := hook.Fire(entry); err != nil {
return err
}
}
return nil
}
package logrus
import (
"encoding/json"
"fmt"
)
type JSONFormatter struct {
// TimestampFormat sets the format used for marshaling timestamps.
TimestampFormat string
}
func (f *JSONFormatter) Format(entry *Entry) ([]byte, error) {
data := make(Fields, len(entry.Data)+3)
for k, v := range entry.Data {
switch v := v.(type) {
case error:
// Otherwise errors are ignored by `encoding/json`
// https://github.com/Sirupsen/logrus/issues/137
data[k] = v.Error()
default:
data[k] = v
}
}
prefixFieldClashes(data)
if f.TimestampFormat == "" {
f.TimestampFormat = DefaultTimestampFormat
}
data["time"] = entry.Time.Format(f.TimestampFormat)
data["msg"] = entry.Message
data["level"] = entry.Level.String()
serialized, err := json.Marshal(data)
if err != nil {
return nil, fmt.Errorf("Failed to marshal fields to JSON, %v", err)
}
return append(serialized, '\n'), nil
}
package logrus
import (
"encoding/json"
"errors"
"testing"
)
func TestErrorNotLost(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("error", errors.New("wild walrus")))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
entry := make(map[string]interface{})
err = json.Unmarshal(b, &entry)
if err != nil {
t.Fatal("Unable to unmarshal formatted entry: ", err)
}
if entry["error"] != "wild walrus" {
t.Fatal("Error field not set")
}
}
func TestErrorNotLostOnFieldNotNamedError(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("omg", errors.New("wild walrus")))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
entry := make(map[string]interface{})
err = json.Unmarshal(b, &entry)
if err != nil {
t.Fatal("Unable to unmarshal formatted entry: ", err)
}
if entry["omg"] != "wild walrus" {
t.Fatal("Error field not set")
}
}
func TestFieldClashWithTime(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("time", "right now!"))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
entry := make(map[string]interface{})
err = json.Unmarshal(b, &entry)
if err != nil {
t.Fatal("Unable to unmarshal formatted entry: ", err)
}
if entry["fields.time"] != "right now!" {
t.Fatal("fields.time not set to original time field")
}
if entry["time"] != "0001-01-01T00:00:00Z" {
t.Fatal("time field not set to current time, was: ", entry["time"])
}
}
func TestFieldClashWithMsg(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("msg", "something"))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
entry := make(map[string]interface{})
err = json.Unmarshal(b, &entry)
if err != nil {
t.Fatal("Unable to unmarshal formatted entry: ", err)
}
if entry["fields.msg"] != "something" {
t.Fatal("fields.msg not set to original msg field")
}
}
func TestFieldClashWithLevel(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("level", "something"))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
entry := make(map[string]interface{})
err = json.Unmarshal(b, &entry)
if err != nil {
t.Fatal("Unable to unmarshal formatted entry: ", err)
}
if entry["fields.level"] != "something" {
t.Fatal("fields.level not set to original level field")
}
}
func TestJSONEntryEndsWithNewline(t *testing.T) {
formatter := &JSONFormatter{}
b, err := formatter.Format(WithField("level", "something"))
if err != nil {
t.Fatal("Unable to format entry: ", err)
}
if b[len(b)-1] != '\n' {
t.Fatal("Expected JSON log entry to end with a newline")
}
}
package logrus
import (
"io"
"os"
"sync"
)
type Logger struct {
// The logs are `io.Copy`'d to this in a mutex. It's common to set this to a
// file, or leave it default which is `os.Stdout`. You can also set this to
// something more adventorous, such as logging to Kafka.
Out io.Writer
// Hooks for the logger instance. These allow firing events based on logging
// levels and log entries. For example, to send errors to an error tracking
// service, log to StatsD or dump the core on fatal errors.
Hooks levelHooks
// All log entries pass through the formatter before logged to Out. The
// included formatters are `TextFormatter` and `JSONFormatter` for which
// TextFormatter is the default. In development (when a TTY is attached) it
// logs with colors, but to a file it wouldn't. You can easily implement your
// own that implements the `Formatter` interface, see the `README` or included
// formatters for examples.
Formatter Formatter
// The logging level the logger should log at. This is typically (and defaults
// to) `logrus.Info`, which allows Info(), Warn(), Error() and Fatal() to be
// logged. `logrus.Debug` is useful in
Level Level
// Used to sync writing to the log.
mu sync.Mutex
}
// Creates a new logger. Configuration should be set by changing `Formatter`,
// `Out` and `Hooks` directly on the default logger instance. You can also just
// instantiate your own:
//
// var log = &Logger{
// Out: os.Stderr,
// Formatter: new(JSONFormatter),
// Hooks: make(levelHooks),
// Level: logrus.DebugLevel,
// }
//
// It's recommended to make this a global instance called `log`.
func New() *Logger {
return &Logger{
Out: os.Stderr,
Formatter: new(TextFormatter),
Hooks: make(levelHooks),
Level: InfoLevel,
}
}
// Adds a field to the log entry, note that you it doesn't log until you call
// Debug, Print, Info, Warn, Fatal or Panic. It only creates a log entry.
// Ff you want multiple fields, use `WithFields`.
func (logger *Logger) WithField(key string, value interface{}) *Entry {
return NewEntry(logger).WithField(key, value)
}
// Adds a struct of fields to the log entry. All it does is call `WithField` for
// each `Field`.
func (logger *Logger) WithFields(fields Fields) *Entry {
return NewEntry(logger).WithFields(fields)
}
func (logger *Logger) Debugf(format string, args ...interface{}) {
if logger.Level >= DebugLevel {
NewEntry(logger).Debugf(format, args...)
}
}
func (logger *Logger) Infof(format string, args ...interface{}) {
if logger.Level >= InfoLevel {
NewEntry(logger).Infof(format, args...)
}
}
func (logger *Logger) Printf(format string, args ...interface{}) {
NewEntry(logger).Printf(format, args...)
}
func (logger *Logger) Warnf(format string, args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warnf(format, args...)
}
}
func (logger *Logger) Warningf(format string, args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warnf(format, args...)
}
}
func (logger *Logger) Errorf(format string, args ...interface{}) {
if logger.Level >= ErrorLevel {
NewEntry(logger).Errorf(format, args...)
}
}
func (logger *Logger) Fatalf(format string, args ...interface{}) {
if logger.Level >= FatalLevel {
NewEntry(logger).Fatalf(format, args...)
}
}
func (logger *Logger) Panicf(format string, args ...interface{}) {
if logger.Level >= PanicLevel {
NewEntry(logger).Panicf(format, args...)
}
}
func (logger *Logger) Debug(args ...interface{}) {
if logger.Level >= DebugLevel {
NewEntry(logger).Debug(args...)
}
}
func (logger *Logger) Info(args ...interface{}) {
if logger.Level >= InfoLevel {
NewEntry(logger).Info(args...)
}
}
func (logger *Logger) Print(args ...interface{}) {
NewEntry(logger).Info(args...)
}
func (logger *Logger) Warn(args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warn(args...)
}
}
func (logger *Logger) Warning(args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warn(args...)
}
}
func (logger *Logger) Error(args ...interface{}) {
if logger.Level >= ErrorLevel {
NewEntry(logger).Error(args...)
}
}
func (logger *Logger) Fatal(args ...interface{}) {
if logger.Level >= FatalLevel {
NewEntry(logger).Fatal(args...)
}
}
func (logger *Logger) Panic(args ...interface{}) {
if logger.Level >= PanicLevel {
NewEntry(logger).Panic(args...)
}
}
func (logger *Logger) Debugln(args ...interface{}) {
if logger.Level >= DebugLevel {
NewEntry(logger).Debugln(args...)
}
}
func (logger *Logger) Infoln(args ...interface{}) {
if logger.Level >= InfoLevel {
NewEntry(logger).Infoln(args...)
}
}
func (logger *Logger) Println(args ...interface{}) {
NewEntry(logger).Println(args...)
}
func (logger *Logger) Warnln(args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warnln(args...)
}
}
func (logger *Logger) Warningln(args ...interface{}) {
if logger.Level >= WarnLevel {
NewEntry(logger).Warnln(args...)
}
}
func (logger *Logger) Errorln(args ...interface{}) {
if logger.Level >= ErrorLevel {
NewEntry(logger).Errorln(args...)
}
}
func (logger *Logger) Fatalln(args ...interface{}) {
if logger.Level >= FatalLevel {
NewEntry(logger).Fatalln(args...)
}
}
func (logger *Logger) Panicln(args ...interface{}) {
if logger.Level >= PanicLevel {
NewEntry(logger).Panicln(args...)
}
}
package logrus
import (
"fmt"
"log"
)
// Fields type, used to pass to `WithFields`.
type Fields map[string]interface{}
// Level type
type Level uint8
// Convert the Level to a string. E.g. PanicLevel becomes "panic".
func (level Level) String() string {
switch level {
case DebugLevel:
return "debug"
case InfoLevel:
return "info"
case WarnLevel:
return "warning"
case ErrorLevel:
return "error"
case FatalLevel:
return "fatal"
case PanicLevel:
return "panic"
}
return "unknown"
}
// ParseLevel takes a string level and returns the Logrus log level constant.
func ParseLevel(lvl string) (Level, error) {
switch lvl {
case "panic":
return PanicLevel, nil
case "fatal":
return FatalLevel, nil
case "error":
return ErrorLevel, nil
case "warn", "warning":
return WarnLevel, nil
case "info":
return InfoLevel, nil
case "debug":
return DebugLevel, nil
}
var l Level
return l, fmt.Errorf("not a valid logrus Level: %q", lvl)
}
// These are the different logging levels. You can set the logging level to log
// on your instance of logger, obtained with `logrus.New()`.
const (
// PanicLevel level, highest level of severity. Logs and then calls panic with the
// message passed to Debug, Info, ...
PanicLevel Level = iota
// FatalLevel level. Logs and then calls `os.Exit(1)`. It will exit even if the
// logging level is set to Panic.
FatalLevel
// ErrorLevel level. Logs. Used for errors that should definitely be noted.
// Commonly used for hooks to send errors to an error tracking service.
ErrorLevel
// WarnLevel level. Non-critical entries that deserve eyes.
WarnLevel
// InfoLevel level. General operational entries about what's going on inside the
// application.
InfoLevel
// DebugLevel level. Usually only enabled when debugging. Very verbose logging.
DebugLevel
)
// Won't compile if StdLogger can't be realized by a log.Logger
var _ StdLogger = &log.Logger{}
// StdLogger is what your logrus-enabled library should take, that way
// it'll accept a stdlib logger and a logrus logger. There's no standard
// interface, this is the closest we get, unfortunately.
type StdLogger interface {
Print(...interface{})
Printf(string, ...interface{})
Println(...interface{})
Fatal(...interface{})
Fatalf(string, ...interface{})
Fatalln(...interface{})
Panic(...interface{})
Panicf(string, ...interface{})
Panicln(...interface{})
}
package logrus
import (
"bytes"
"encoding/json"
"strconv"
"strings"
"sync"
"testing"
"github.com/fsouza/go-dockerclient/vendor/github.com/stretchr/testify/assert"
)
func LogAndAssertJSON(t *testing.T, log func(*Logger), assertions func(fields Fields)) {
var buffer bytes.Buffer
var fields Fields
logger := New()
logger.Out = &buffer
logger.Formatter = new(JSONFormatter)
log(logger)
err := json.Unmarshal(buffer.Bytes(), &fields)
assert.Nil(t, err)
assertions(fields)
}
func LogAndAssertText(t *testing.T, log func(*Logger), assertions func(fields map[string]string)) {
var buffer bytes.Buffer
logger := New()
logger.Out = &buffer
logger.Formatter = &TextFormatter{
DisableColors: true,
}
log(logger)
fields := make(map[string]string)
for _, kv := range strings.Split(buffer.String(), " ") {
if !strings.Contains(kv, "=") {
continue
}
kvArr := strings.Split(kv, "=")
key := strings.TrimSpace(kvArr[0])
val := kvArr[1]
if kvArr[1][0] == '"' {
var err error
val, err = strconv.Unquote(val)
assert.NoError(t, err)
}
fields[key] = val
}
assertions(fields)
}
func TestPrint(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Print("test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test")
assert.Equal(t, fields["level"], "info")
})
}
func TestInfo(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Info("test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test")
assert.Equal(t, fields["level"], "info")
})
}
func TestWarn(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Warn("test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test")
assert.Equal(t, fields["level"], "warning")
})
}
func TestInfolnShouldAddSpacesBetweenStrings(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Infoln("test", "test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test test")
})
}
func TestInfolnShouldAddSpacesBetweenStringAndNonstring(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Infoln("test", 10)
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test 10")
})
}
func TestInfolnShouldAddSpacesBetweenTwoNonStrings(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Infoln(10, 10)
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "10 10")
})
}
func TestInfoShouldAddSpacesBetweenTwoNonStrings(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Infoln(10, 10)
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "10 10")
})
}
func TestInfoShouldNotAddSpacesBetweenStringAndNonstring(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Info("test", 10)
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test10")
})
}
func TestInfoShouldNotAddSpacesBetweenStrings(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.Info("test", "test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "testtest")
})
}
func TestWithFieldsShouldAllowAssignments(t *testing.T) {
var buffer bytes.Buffer
var fields Fields
logger := New()
logger.Out = &buffer
logger.Formatter = new(JSONFormatter)
localLog := logger.WithFields(Fields{
"key1": "value1",
})
localLog.WithField("key2", "value2").Info("test")
err := json.Unmarshal(buffer.Bytes(), &fields)
assert.Nil(t, err)
assert.Equal(t, "value2", fields["key2"])
assert.Equal(t, "value1", fields["key1"])
buffer = bytes.Buffer{}
fields = Fields{}
localLog.Info("test")
err = json.Unmarshal(buffer.Bytes(), &fields)
assert.Nil(t, err)
_, ok := fields["key2"]
assert.Equal(t, false, ok)
assert.Equal(t, "value1", fields["key1"])
}
func TestUserSuppliedFieldDoesNotOverwriteDefaults(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.WithField("msg", "hello").Info("test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test")
})
}
func TestUserSuppliedMsgFieldHasPrefix(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.WithField("msg", "hello").Info("test")
}, func(fields Fields) {
assert.Equal(t, fields["msg"], "test")
assert.Equal(t, fields["fields.msg"], "hello")
})
}
func TestUserSuppliedTimeFieldHasPrefix(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.WithField("time", "hello").Info("test")
}, func(fields Fields) {
assert.Equal(t, fields["fields.time"], "hello")
})
}
func TestUserSuppliedLevelFieldHasPrefix(t *testing.T) {
LogAndAssertJSON(t, func(log *Logger) {
log.WithField("level", 1).Info("test")
}, func(fields Fields) {
assert.Equal(t, fields["level"], "info")
assert.Equal(t, fields["fields.level"], 1.0) // JSON has floats only
})
}
func TestDefaultFieldsAreNotPrefixed(t *testing.T) {
LogAndAssertText(t, func(log *Logger) {
ll := log.WithField("herp", "derp")
ll.Info("hello")
ll.Info("bye")
}, func(fields map[string]string) {
for _, fieldName := range []string{"fields.level", "fields.time", "fields.msg"} {
if _, ok := fields[fieldName]; ok {
t.Fatalf("should not have prefixed %q: %v", fieldName, fields)
}
}
})
}
func TestDoubleLoggingDoesntPrefixPreviousFields(t *testing.T) {
var buffer bytes.Buffer
var fields Fields
logger := New()
logger.Out = &buffer
logger.Formatter = new(JSONFormatter)
llog := logger.WithField("context", "eating raw fish")
llog.Info("looks delicious")
err := json.Unmarshal(buffer.Bytes(), &fields)
assert.NoError(t, err, "should have decoded first message")
assert.Equal(t, len(fields), 4, "should only have msg/time/level/context fields")
assert.Equal(t, fields["msg"], "looks delicious")
assert.Equal(t, fields["context"], "eating raw fish")
buffer.Reset()
llog.Warn("omg it is!")
err = json.Unmarshal(buffer.Bytes(), &fields)
assert.NoError(t, err, "should have decoded second message")
assert.Equal(t, len(fields), 4, "should only have msg/time/level/context fields")
assert.Equal(t, fields["msg"], "omg it is!")
assert.Equal(t, fields["context"], "eating raw fish")
assert.Nil(t, fields["fields.msg"], "should not have prefixed previous `msg` entry")
}
func TestConvertLevelToString(t *testing.T) {
assert.Equal(t, "debug", DebugLevel.String())
assert.Equal(t, "info", InfoLevel.String())
assert.Equal(t, "warning", WarnLevel.String())
assert.Equal(t, "error", ErrorLevel.String())
assert.Equal(t, "fatal", FatalLevel.String())
assert.Equal(t, "panic", PanicLevel.String())
}
func TestParseLevel(t *testing.T) {
l, err := ParseLevel("panic")
assert.Nil(t, err)
assert.Equal(t, PanicLevel, l)
l, err = ParseLevel("fatal")
assert.Nil(t, err)
assert.Equal(t, FatalLevel, l)
l, err = ParseLevel("error")
assert.Nil(t, err)
assert.Equal(t, ErrorLevel, l)
l, err = ParseLevel("warn")
assert.Nil(t, err)
assert.Equal(t, WarnLevel, l)
l, err = ParseLevel("warning")
assert.Nil(t, err)
assert.Equal(t, WarnLevel, l)
l, err = ParseLevel("info")
assert.Nil(t, err)
assert.Equal(t, InfoLevel, l)
l, err = ParseLevel("debug")
assert.Nil(t, err)
assert.Equal(t, DebugLevel, l)
l, err = ParseLevel("invalid")
assert.Equal(t, "not a valid logrus Level: \"invalid\"", err.Error())
}
func TestGetSetLevelRace(t *testing.T) {
wg := sync.WaitGroup{}
for i := 0; i < 100; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
if i%2 == 0 {
SetLevel(InfoLevel)
} else {
GetLevel()
}
}(i)
}
wg.Wait()
}
// Based on ssh/terminal:
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package logrus
import "syscall"
const ioctlReadTermios = syscall.TIOCGETA
type Termios syscall.Termios
/*
Go 1.2 doesn't include Termios for FreeBSD. This should be added in 1.3 and this could be merged with terminal_darwin.
*/
package logrus
import (
"syscall"
)
const ioctlReadTermios = syscall.TIOCGETA
type Termios struct {
Iflag uint32
Oflag uint32
Cflag uint32
Lflag uint32
Cc [20]uint8
Ispeed uint32
Ospeed uint32
}
// Based on ssh/terminal:
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package logrus
import "syscall"
const ioctlReadTermios = syscall.TCGETS
type Termios syscall.Termios
// Based on ssh/terminal:
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build linux darwin freebsd openbsd
package logrus
import (
"syscall"
"unsafe"
)
// IsTerminal returns true if the given file descriptor is a terminal.
func IsTerminal() bool {
fd := syscall.Stdout
var termios Termios
_, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), ioctlReadTermios, uintptr(unsafe.Pointer(&termios)), 0, 0, 0)
return err == 0
}
package logrus
import "syscall"
const ioctlReadTermios = syscall.TIOCGETA
type Termios syscall.Termios
// Based on ssh/terminal:
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package logrus
import (
"syscall"
"unsafe"
)
var kernel32 = syscall.NewLazyDLL("kernel32.dll")
var (
procGetConsoleMode = kernel32.NewProc("GetConsoleMode")
)
// IsTerminal returns true if the given file descriptor is a terminal.
func IsTerminal() bool {
fd := syscall.Stdout
var st uint32
r, _, e := syscall.Syscall(procGetConsoleMode.Addr(), 2, uintptr(fd), uintptr(unsafe.Pointer(&st)), 0)
return r != 0 && e == 0
}
package logrus
import (
"bytes"
"fmt"
"sort"
"strings"
"time"
)
const (
nocolor = 0
red = 31
green = 32
yellow = 33
blue = 34
gray = 37
)
var (
baseTimestamp time.Time
isTerminal bool
)
func init() {
baseTimestamp = time.Now()
isTerminal = IsTerminal()
}
func miniTS() int {
return int(time.Since(baseTimestamp) / time.Second)
}
type TextFormatter struct {
// Set to true to bypass checking for a TTY before outputting colors.
ForceColors bool
// Force disabling colors.
DisableColors bool
// Disable timestamp logging. useful when output is redirected to logging
// system that already adds timestamps.
DisableTimestamp bool
// Enable logging the full timestamp when a TTY is attached instead of just
// the time passed since beginning of execution.
FullTimestamp bool
// TimestampFormat to use for display when a full timestamp is printed
TimestampFormat string
// The fields are sorted by default for a consistent output. For applications
// that log extremely frequently and don't use the JSON formatter this may not
// be desired.
DisableSorting bool
}
func (f *TextFormatter) Format(entry *Entry) ([]byte, error) {
var keys []string = make([]string, 0, len(entry.Data))
for k := range entry.Data {
keys = append(keys, k)
}
if !f.DisableSorting {
sort.Strings(keys)
}
b := &bytes.Buffer{}
prefixFieldClashes(entry.Data)
isColored := (f.ForceColors || isTerminal) && !f.DisableColors
if f.TimestampFormat == "" {
f.TimestampFormat = DefaultTimestampFormat
}
if isColored {
f.printColored(b, entry, keys)
} else {
if !f.DisableTimestamp {
f.appendKeyValue(b, "time", entry.Time.Format(f.TimestampFormat))
}
f.appendKeyValue(b, "level", entry.Level.String())
f.appendKeyValue(b, "msg", entry.Message)
for _, key := range keys {
f.appendKeyValue(b, key, entry.Data[key])
}
}
b.WriteByte('\n')
return b.Bytes(), nil
}
func (f *TextFormatter) printColored(b *bytes.Buffer, entry *Entry, keys []string) {
var levelColor int
switch entry.Level {
case DebugLevel:
levelColor = gray
case WarnLevel:
levelColor = yellow
case ErrorLevel, FatalLevel, PanicLevel:
levelColor = red
default:
levelColor = blue
}
levelText := strings.ToUpper(entry.Level.String())[0:4]
if !f.FullTimestamp {
fmt.Fprintf(b, "\x1b[%dm%s\x1b[0m[%04d] %-44s ", levelColor, levelText, miniTS(), entry.Message)
} else {
fmt.Fprintf(b, "\x1b[%dm%s\x1b[0m[%s] %-44s ", levelColor, levelText, entry.Time.Format(f.TimestampFormat), entry.Message)
}
for _, k := range keys {
v := entry.Data[k]
fmt.Fprintf(b, " \x1b[%dm%s\x1b[0m=%v", levelColor, k, v)
}
}
func needsQuoting(text string) bool {
for _, ch := range text {
if !((ch >= 'a' && ch <= 'z') ||
(ch >= 'A' && ch <= 'Z') ||
(ch >= '0' && ch <= '9') ||
ch == '-' || ch == '.') {
return false
}
}
return true
}
func (f *TextFormatter) appendKeyValue(b *bytes.Buffer, key, value interface{}) {
switch value.(type) {
case string:
if needsQuoting(value.(string)) {
fmt.Fprintf(b, "%v=%s ", key, value)
} else {
fmt.Fprintf(b, "%v=%q ", key, value)
}
case error:
if needsQuoting(value.(error).Error()) {
fmt.Fprintf(b, "%v=%s ", key, value)
} else {
fmt.Fprintf(b, "%v=%q ", key, value)
}
default:
fmt.Fprintf(b, "%v=%v ", key, value)
}
}
package logrus
import (
"bytes"
"errors"
"testing"
"time"
)
func TestQuoting(t *testing.T) {
tf := &TextFormatter{DisableColors: true}
checkQuoting := func(q bool, value interface{}) {
b, _ := tf.Format(WithField("test", value))
idx := bytes.Index(b, ([]byte)("test="))
cont := bytes.Contains(b[idx+5:], []byte{'"'})
if cont != q {
if q {
t.Errorf("quoting expected for: %#v", value)
} else {
t.Errorf("quoting not expected for: %#v", value)
}
}
}
checkQuoting(false, "abcd")
checkQuoting(false, "v1.0")
checkQuoting(false, "1234567890")
checkQuoting(true, "/foobar")
checkQuoting(true, "x y")
checkQuoting(true, "x,y")
checkQuoting(false, errors.New("invalid"))
checkQuoting(true, errors.New("invalid argument"))
}
func TestTimestampFormat(t *testing.T) {
checkTimeStr := func(format string) {
customFormatter := &TextFormatter{DisableColors: true, TimestampFormat: format}
customStr, _ := customFormatter.Format(WithField("test", "test"))
timeStart := bytes.Index(customStr, ([]byte)("time="))
timeEnd := bytes.Index(customStr, ([]byte)("level="))
timeStr := customStr[timeStart+5 : timeEnd-1]
if timeStr[0] == '"' && timeStr[len(timeStr)-1] == '"' {
timeStr = timeStr[1 : len(timeStr)-1]
}
if format == "" {
format = time.RFC3339
}
_, e := time.Parse(format, (string)(timeStr))
if e != nil {
t.Errorf("time string \"%s\" did not match provided time format \"%s\": %s", timeStr, format, e)
}
}
checkTimeStr("2006-01-02T15:04:05.000000000Z07:00")
checkTimeStr("Mon Jan _2 15:04:05 2006")
checkTimeStr("")
}
// TODO add tests for sorting etc., this requires a parser for the text
// formatter output.
package logrus
import (
"bufio"
"io"
"runtime"
)
func (logger *Logger) Writer() *io.PipeWriter {
reader, writer := io.Pipe()
go logger.writerScanner(reader)
runtime.SetFinalizer(writer, writerFinalizer)
return writer
}
func (logger *Logger) writerScanner(reader *io.PipeReader) {
scanner := bufio.NewScanner(reader)
for scanner.Scan() {
logger.Print(scanner.Text())
}
if err := scanner.Err(); err != nil {
logger.Errorf("Error while reading from Writer: %s", err)
}
reader.Close()
}
func writerFinalizer(writer *io.PipeWriter) {
writer.Close()
}
// +build !windows
package archive
import (
"archive/tar"
"errors"
"os"
"syscall"
)
// canonicalTarNameForPath returns platform-specific filepath
// to canonical posix-style path for tar archival. p is relative
// path.
func CanonicalTarNameForPath(p string) (string, error) {
return p, nil // already unix-style
}
// chmodTarEntry is used to adjust the file permissions used in tar header based
// on the platform the archival is done.
func chmodTarEntry(perm os.FileMode) os.FileMode {
return perm // noop for unix as golang APIs provide perm bits correctly
}
func setHeaderForSpecialDevice(hdr *tar.Header, ta *tarAppender, name string, stat interface{}) (nlink uint32, inode uint64, err error) {
s, ok := stat.(*syscall.Stat_t)
if !ok {
err = errors.New("cannot convert stat value to syscall.Stat_t")
return
}
nlink = uint32(s.Nlink)
inode = uint64(s.Ino)
// Currently go does not fil in the major/minors
if s.Mode&syscall.S_IFBLK != 0 ||
s.Mode&syscall.S_IFCHR != 0 {
hdr.Devmajor = int64(major(uint64(s.Rdev)))
hdr.Devminor = int64(minor(uint64(s.Rdev)))
}
return
}
func major(device uint64) uint64 {
return (device >> 8) & 0xfff
}
func minor(device uint64) uint64 {
return (device & 0xff) | ((device >> 12) & 0xfff00)
}
// +build !windows
package archive
import (
"os"
"testing"
)
func TestCanonicalTarNameForPath(t *testing.T) {
cases := []struct{ in, expected string }{
{"foo", "foo"},
{"foo/bar", "foo/bar"},
{"foo/dir/", "foo/dir/"},
}
for _, v := range cases {
if out, err := CanonicalTarNameForPath(v.in); err != nil {
t.Fatalf("cannot get canonical name for path: %s: %v", v.in, err)
} else if out != v.expected {
t.Fatalf("wrong canonical tar name. expected:%s got:%s", v.expected, out)
}
}
}
func TestCanonicalTarName(t *testing.T) {
cases := []struct {
in string
isDir bool
expected string
}{
{"foo", false, "foo"},
{"foo", true, "foo/"},
{"foo/bar", false, "foo/bar"},
{"foo/bar", true, "foo/bar/"},
}
for _, v := range cases {
if out, err := canonicalTarName(v.in, v.isDir); err != nil {
t.Fatalf("cannot get canonical name for path: %s: %v", v.in, err)
} else if out != v.expected {
t.Fatalf("wrong canonical tar name. expected:%s got:%s", v.expected, out)
}
}
}
func TestChmodTarEntry(t *testing.T) {
cases := []struct {
in, expected os.FileMode
}{
{0000, 0000},
{0777, 0777},
{0644, 0644},
{0755, 0755},
{0444, 0444},
}
for _, v := range cases {
if out := chmodTarEntry(v.in); out != v.expected {
t.Fatalf("wrong chmod. expected:%v got:%v", v.expected, out)
}
}
}
// +build windows
package archive
import (
"archive/tar"
"fmt"
"os"
"strings"
)
// canonicalTarNameForPath returns platform-specific filepath
// to canonical posix-style path for tar archival. p is relative
// path.
func CanonicalTarNameForPath(p string) (string, error) {
// windows: convert windows style relative path with backslashes
// into forward slashes. since windows does not allow '/' or '\'
// in file names, it is mostly safe to replace however we must
// check just in case
if strings.Contains(p, "/") {
return "", fmt.Errorf("windows path contains forward slash: %s", p)
}
return strings.Replace(p, string(os.PathSeparator), "/", -1), nil
}
// chmodTarEntry is used to adjust the file permissions used in tar header based
// on the platform the archival is done.
func chmodTarEntry(perm os.FileMode) os.FileMode {
perm &= 0755
// Add the x bit: make everything +x from windows
perm |= 0111
return perm
}
func setHeaderForSpecialDevice(hdr *tar.Header, ta *tarAppender, name string, stat interface{}) (nlink uint32, inode uint64, err error) {
// do nothing. no notion of Rdev, Inode, Nlink in stat on Windows
return
}
// +build windows
package archive
import (
"os"
"testing"
)
func TestCanonicalTarNameForPath(t *testing.T) {
cases := []struct {
in, expected string
shouldFail bool
}{
{"foo", "foo", false},
{"foo/bar", "___", true}, // unix-styled windows path must fail
{`foo\bar`, "foo/bar", false},
}
for _, v := range cases {
if out, err := CanonicalTarNameForPath(v.in); err != nil && !v.shouldFail {
t.Fatalf("cannot get canonical name for path: %s: %v", v.in, err)
} else if v.shouldFail && err == nil {
t.Fatalf("canonical path call should have failed with error. in=%s out=%s", v.in, out)
} else if !v.shouldFail && out != v.expected {
t.Fatalf("wrong canonical tar name. expected:%s got:%s", v.expected, out)
}
}
}
func TestCanonicalTarName(t *testing.T) {
cases := []struct {
in string
isDir bool
expected string
}{
{"foo", false, "foo"},
{"foo", true, "foo/"},
{`foo\bar`, false, "foo/bar"},
{`foo\bar`, true, "foo/bar/"},
}
for _, v := range cases {
if out, err := canonicalTarName(v.in, v.isDir); err != nil {
t.Fatalf("cannot get canonical name for path: %s: %v", v.in, err)
} else if out != v.expected {
t.Fatalf("wrong canonical tar name. expected:%s got:%s", v.expected, out)
}
}
}
func TestChmodTarEntry(t *testing.T) {
cases := []struct {
in, expected os.FileMode
}{
{0000, 0111},
{0777, 0755},
{0644, 0755},
{0755, 0755},
{0444, 0555},
}
for _, v := range cases {
if out := chmodTarEntry(v.in); out != v.expected {
t.Fatalf("wrong chmod. expected:%v got:%v", v.expected, out)
}
}
}
package archive
import (
"archive/tar"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"sort"
"testing"
)
func TestHardLinkOrder(t *testing.T) {
names := []string{"file1.txt", "file2.txt", "file3.txt"}
msg := []byte("Hey y'all")
// Create dir
src, err := ioutil.TempDir("", "docker-hardlink-test-src-")
if err != nil {
t.Fatal(err)
}
//defer os.RemoveAll(src)
for _, name := range names {
func() {
fh, err := os.Create(path.Join(src, name))
if err != nil {
t.Fatal(err)
}
defer fh.Close()
if _, err = fh.Write(msg); err != nil {
t.Fatal(err)
}
}()
}
// Create dest, with changes that includes hardlinks
dest, err := ioutil.TempDir("", "docker-hardlink-test-dest-")
if err != nil {
t.Fatal(err)
}
os.RemoveAll(dest) // we just want the name, at first
if err := copyDir(src, dest); err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dest)
for _, name := range names {
for i := 0; i < 5; i++ {
if err := os.Link(path.Join(dest, name), path.Join(dest, fmt.Sprintf("%s.link%d", name, i))); err != nil {
t.Fatal(err)
}
}
}
// get changes
changes, err := ChangesDirs(dest, src)
if err != nil {
t.Fatal(err)
}
// sort
sort.Sort(changesByPath(changes))
// ExportChanges
ar, err := ExportChanges(dest, changes)
if err != nil {
t.Fatal(err)
}
hdrs, err := walkHeaders(ar)
if err != nil {
t.Fatal(err)
}
// reverse sort
sort.Sort(sort.Reverse(changesByPath(changes)))
// ExportChanges
arRev, err := ExportChanges(dest, changes)
if err != nil {
t.Fatal(err)
}
hdrsRev, err := walkHeaders(arRev)
if err != nil {
t.Fatal(err)
}
// line up the two sets
sort.Sort(tarHeaders(hdrs))
sort.Sort(tarHeaders(hdrsRev))
// compare Size and LinkName
for i := range hdrs {
if hdrs[i].Name != hdrsRev[i].Name {
t.Errorf("headers - expected name %q; but got %q", hdrs[i].Name, hdrsRev[i].Name)
}
if hdrs[i].Size != hdrsRev[i].Size {
t.Errorf("headers - %q expected size %d; but got %d", hdrs[i].Name, hdrs[i].Size, hdrsRev[i].Size)
}
if hdrs[i].Typeflag != hdrsRev[i].Typeflag {
t.Errorf("headers - %q expected type %d; but got %d", hdrs[i].Name, hdrs[i].Typeflag, hdrsRev[i].Typeflag)
}
if hdrs[i].Linkname != hdrsRev[i].Linkname {
t.Errorf("headers - %q expected linkname %q; but got %q", hdrs[i].Name, hdrs[i].Linkname, hdrsRev[i].Linkname)
}
}
}
type tarHeaders []tar.Header
func (th tarHeaders) Len() int { return len(th) }
func (th tarHeaders) Swap(i, j int) { th[j], th[i] = th[i], th[j] }
func (th tarHeaders) Less(i, j int) bool { return th[i].Name < th[j].Name }
func walkHeaders(r io.Reader) ([]tar.Header, error) {
t := tar.NewReader(r)
headers := []tar.Header{}
for {
hdr, err := t.Next()
if err != nil {
if err == io.EOF {
break
}
return headers, err
}
headers = append(headers, *hdr)
}
return headers, nil
}
package archive
import (
"archive/tar"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/pools"
"github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/system"
)
func UnpackLayer(dest string, layer ArchiveReader) (size int64, err error) {
tr := tar.NewReader(layer)
trBuf := pools.BufioReader32KPool.Get(tr)
defer pools.BufioReader32KPool.Put(trBuf)
var dirs []*tar.Header
aufsTempdir := ""
aufsHardlinks := make(map[string]*tar.Header)
// Iterate through the files in the archive.
for {
hdr, err := tr.Next()
if err == io.EOF {
// end of tar archive
break
}
if err != nil {
return 0, err
}
size += hdr.Size
// Normalize name, for safety and for a simple is-root check
hdr.Name = filepath.Clean(hdr.Name)
if !strings.HasSuffix(hdr.Name, "/") {
// Not the root directory, ensure that the parent directory exists.
// This happened in some tests where an image had a tarfile without any
// parent directories.
parent := filepath.Dir(hdr.Name)
parentPath := filepath.Join(dest, parent)
if _, err := os.Lstat(parentPath); err != nil && os.IsNotExist(err) {
err = os.MkdirAll(parentPath, 0600)
if err != nil {
return 0, err
}
}
}
// Skip AUFS metadata dirs
if strings.HasPrefix(hdr.Name, ".wh..wh.") {
// Regular files inside /.wh..wh.plnk can be used as hardlink targets
// We don't want this directory, but we need the files in them so that
// such hardlinks can be resolved.
if strings.HasPrefix(hdr.Name, ".wh..wh.plnk") && hdr.Typeflag == tar.TypeReg {
basename := filepath.Base(hdr.Name)
aufsHardlinks[basename] = hdr
if aufsTempdir == "" {
if aufsTempdir, err = ioutil.TempDir("", "dockerplnk"); err != nil {
return 0, err
}
defer os.RemoveAll(aufsTempdir)
}
if err := createTarFile(filepath.Join(aufsTempdir, basename), dest, hdr, tr, true); err != nil {
return 0, err
}
}
continue
}
path := filepath.Join(dest, hdr.Name)
rel, err := filepath.Rel(dest, path)
if err != nil {
return 0, err
}
if strings.HasPrefix(rel, "../") {
return 0, breakoutError(fmt.Errorf("%q is outside of %q", hdr.Name, dest))
}
base := filepath.Base(path)
if strings.HasPrefix(base, ".wh.") {
originalBase := base[len(".wh."):]
originalPath := filepath.Join(filepath.Dir(path), originalBase)
if err := os.RemoveAll(originalPath); err != nil {
return 0, err
}
} else {
// If path exits we almost always just want to remove and replace it.
// The only exception is when it is a directory *and* the file from
// the layer is also a directory. Then we want to merge them (i.e.
// just apply the metadata from the layer).
if fi, err := os.Lstat(path); err == nil {
if !(fi.IsDir() && hdr.Typeflag == tar.TypeDir) {
if err := os.RemoveAll(path); err != nil {
return 0, err
}
}
}
trBuf.Reset(tr)
srcData := io.Reader(trBuf)
srcHdr := hdr
// Hard links into /.wh..wh.plnk don't work, as we don't extract that directory, so
// we manually retarget these into the temporary files we extracted them into
if hdr.Typeflag == tar.TypeLink && strings.HasPrefix(filepath.Clean(hdr.Linkname), ".wh..wh.plnk") {
linkBasename := filepath.Base(hdr.Linkname)
srcHdr = aufsHardlinks[linkBasename]
if srcHdr == nil {
return 0, fmt.Errorf("Invalid aufs hardlink")
}
tmpFile, err := os.Open(filepath.Join(aufsTempdir, linkBasename))
if err != nil {
return 0, err
}
defer tmpFile.Close()
srcData = tmpFile
}
if err := createTarFile(path, dest, srcHdr, srcData, true); err != nil {
return 0, err
}
// Directory mtimes must be handled at the end to avoid further
// file creation in them to modify the directory mtime
if hdr.Typeflag == tar.TypeDir {
dirs = append(dirs, hdr)
}
}
}
for _, hdr := range dirs {
path := filepath.Join(dest, hdr.Name)
ts := []syscall.Timespec{timeToTimespec(hdr.AccessTime), timeToTimespec(hdr.ModTime)}
if err := syscall.UtimesNano(path, ts); err != nil {
return 0, err
}
}
return size, nil
}
// ApplyLayer parses a diff in the standard layer format from `layer`, and
// applies it to the directory `dest`. Returns the size in bytes of the
// contents of the layer.
func ApplyLayer(dest string, layer ArchiveReader) (int64, error) {
dest = filepath.Clean(dest)
// We need to be able to set any perms
oldmask, err := system.Umask(0)
if err != nil {
return 0, err
}
defer system.Umask(oldmask) // ignore err, ErrNotSupportedPlatform
layer, err = DecompressStream(layer)
if err != nil {
return 0, err
}
return UnpackLayer(dest, layer)
}
package archive
import (
"archive/tar"
"testing"
)
func TestApplyLayerInvalidFilenames(t *testing.T) {
for i, headers := range [][]*tar.Header{
{
{
Name: "../victim/dotdot",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
{
{
// Note the leading slash
Name: "/../victim/slash-dotdot",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
} {
if err := testBreakout("applylayer", "docker-TestApplyLayerInvalidFilenames", headers); err != nil {
t.Fatalf("i=%d. %v", i, err)
}
}
}
func TestApplyLayerInvalidHardlink(t *testing.T) {
for i, headers := range [][]*tar.Header{
{ // try reading victim/hello (../)
{
Name: "dotdot",
Typeflag: tar.TypeLink,
Linkname: "../victim/hello",
Mode: 0644,
},
},
{ // try reading victim/hello (/../)
{
Name: "slash-dotdot",
Typeflag: tar.TypeLink,
// Note the leading slash
Linkname: "/../victim/hello",
Mode: 0644,
},
},
{ // try writing victim/file
{
Name: "loophole-victim",
Typeflag: tar.TypeLink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "loophole-victim/file",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
{ // try reading victim/hello (hardlink, symlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeLink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "symlink",
Typeflag: tar.TypeSymlink,
Linkname: "loophole-victim/hello",
Mode: 0644,
},
},
{ // Try reading victim/hello (hardlink, hardlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeLink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "hardlink",
Typeflag: tar.TypeLink,
Linkname: "loophole-victim/hello",
Mode: 0644,
},
},
{ // Try removing victim directory (hardlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeLink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "loophole-victim",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
} {
if err := testBreakout("applylayer", "docker-TestApplyLayerInvalidHardlink", headers); err != nil {
t.Fatalf("i=%d. %v", i, err)
}
}
}
func TestApplyLayerInvalidSymlink(t *testing.T) {
for i, headers := range [][]*tar.Header{
{ // try reading victim/hello (../)
{
Name: "dotdot",
Typeflag: tar.TypeSymlink,
Linkname: "../victim/hello",
Mode: 0644,
},
},
{ // try reading victim/hello (/../)
{
Name: "slash-dotdot",
Typeflag: tar.TypeSymlink,
// Note the leading slash
Linkname: "/../victim/hello",
Mode: 0644,
},
},
{ // try writing victim/file
{
Name: "loophole-victim",
Typeflag: tar.TypeSymlink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "loophole-victim/file",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
{ // try reading victim/hello (symlink, symlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeSymlink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "symlink",
Typeflag: tar.TypeSymlink,
Linkname: "loophole-victim/hello",
Mode: 0644,
},
},
{ // try reading victim/hello (symlink, hardlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeSymlink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "hardlink",
Typeflag: tar.TypeLink,
Linkname: "loophole-victim/hello",
Mode: 0644,
},
},
{ // try removing victim directory (symlink)
{
Name: "loophole-victim",
Typeflag: tar.TypeSymlink,
Linkname: "../victim",
Mode: 0755,
},
{
Name: "loophole-victim",
Typeflag: tar.TypeReg,
Mode: 0644,
},
},
} {
if err := testBreakout("applylayer", "docker-TestApplyLayerInvalidSymlink", headers); err != nil {
t.Fatalf("i=%d. %v", i, err)
}
}
}
// +build ignore
// Simple tool to create an archive stream from an old and new directory
//
// By default it will stream the comparison of two temporary directories with junk files
package main
import (
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"github.com/fsouza/go-dockerclient/vendor/github.com/Sirupsen/logrus"
"github.com/fsouza/go-dockerclient/vendor/github.com/docker/docker/pkg/archive"
)
var (
flDebug = flag.Bool("D", false, "debugging output")
flNewDir = flag.String("newdir", "", "")
flOldDir = flag.String("olddir", "", "")
log = logrus.New()
)
func main() {
flag.Usage = func() {
fmt.Println("Produce a tar from comparing two directory paths. By default a demo tar is created of around 200 files (including hardlinks)")
fmt.Printf("%s [OPTIONS]\n", os.Args[0])
flag.PrintDefaults()
}
flag.Parse()
log.Out = os.Stderr
if (len(os.Getenv("DEBUG")) > 0) || *flDebug {
logrus.SetLevel(logrus.DebugLevel)
}
var newDir, oldDir string
if len(*flNewDir) == 0 {
var err error
newDir, err = ioutil.TempDir("", "docker-test-newDir")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(newDir)
if _, err := prepareUntarSourceDirectory(100, newDir, true); err != nil {
log.Fatal(err)
}
} else {
newDir = *flNewDir
}
if len(*flOldDir) == 0 {
oldDir, err := ioutil.TempDir("", "docker-test-oldDir")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(oldDir)
} else {
oldDir = *flOldDir
}
changes, err := archive.ChangesDirs(newDir, oldDir)
if err != nil {
log.Fatal(err)
}
a, err := archive.ExportChanges(newDir, changes)
if err != nil {
log.Fatal(err)
}
defer a.Close()
i, err := io.Copy(os.Stdout, a)
if err != nil && err != io.EOF {
log.Fatal(err)
}
fmt.Fprintf(os.Stderr, "wrote archive of %d bytes", i)
}
func prepareUntarSourceDirectory(numberOfFiles int, targetPath string, makeLinks bool) (int, error) {
fileData := []byte("fooo")
for n := 0; n < numberOfFiles; n++ {
fileName := fmt.Sprintf("file-%d", n)
if err := ioutil.WriteFile(path.Join(targetPath, fileName), fileData, 0700); err != nil {
return 0, err
}
if makeLinks {
if err := os.Link(path.Join(targetPath, fileName), path.Join(targetPath, fileName+"-link")); err != nil {
return 0, err
}
}
}
totalSize := numberOfFiles * len(fileData)
return totalSize, nil
}
package archive
import (
"syscall"
"time"
)
func timeToTimespec(time time.Time) (ts syscall.Timespec) {
if time.IsZero() {
// Return UTIME_OMIT special value
ts.Sec = 0
ts.Nsec = ((1 << 30) - 2)
return
}
return syscall.NsecToTimespec(time.UnixNano())
}
// +build !linux
package archive
import (
"syscall"
"time"
)
func timeToTimespec(time time.Time) (ts syscall.Timespec) {
nsec := int64(0)
if !time.IsZero() {
nsec = time.UnixNano()
}
return syscall.NsecToTimespec(nsec)
}
package archive
import (
"archive/tar"
"bytes"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"time"
)
var testUntarFns = map[string]func(string, io.Reader) error{
"untar": func(dest string, r io.Reader) error {
return Untar(r, dest, nil)
},
"applylayer": func(dest string, r io.Reader) error {
_, err := ApplyLayer(dest, ArchiveReader(r))
return err
},
}
// testBreakout is a helper function that, within the provided `tmpdir` directory,
// creates a `victim` folder with a generated `hello` file in it.
// `untar` extracts to a directory named `dest`, the tar file created from `headers`.
//
// Here are the tested scenarios:
// - removed `victim` folder (write)
// - removed files from `victim` folder (write)
// - new files in `victim` folder (write)
// - modified files in `victim` folder (write)
// - file in `dest` with same content as `victim/hello` (read)
//
// When using testBreakout make sure you cover one of the scenarios listed above.
func testBreakout(untarFn string, tmpdir string, headers []*tar.Header) error {
tmpdir, err := ioutil.TempDir("", tmpdir)
if err != nil {
return err
}
defer os.RemoveAll(tmpdir)
dest := filepath.Join(tmpdir, "dest")
if err := os.Mkdir(dest, 0755); err != nil {
return err
}
victim := filepath.Join(tmpdir, "victim")
if err := os.Mkdir(victim, 0755); err != nil {
return err
}
hello := filepath.Join(victim, "hello")
helloData, err := time.Now().MarshalText()
if err != nil {
return err
}
if err := ioutil.WriteFile(hello, helloData, 0644); err != nil {
return err
}
helloStat, err := os.Stat(hello)
if err != nil {
return err
}
reader, writer := io.Pipe()
go func() {
t := tar.NewWriter(writer)
for _, hdr := range headers {
t.WriteHeader(hdr)
}
t.Close()
}()
untar := testUntarFns[untarFn]
if untar == nil {
return fmt.Errorf("could not find untar function %q in testUntarFns", untarFn)
}
if err := untar(dest, reader); err != nil {
if _, ok := err.(breakoutError); !ok {
// If untar returns an error unrelated to an archive breakout,
// then consider this an unexpected error and abort.
return err
}
// Here, untar detected the breakout.
// Let's move on verifying that indeed there was no breakout.
fmt.Printf("breakoutError: %v\n", err)
}
// Check victim folder
f, err := os.Open(victim)
if err != nil {
// codepath taken if victim folder was removed
return fmt.Errorf("archive breakout: error reading %q: %v", victim, err)
}
defer f.Close()
// Check contents of victim folder
//
// We are only interested in getting 2 files from the victim folder, because if all is well
// we expect only one result, the `hello` file. If there is a second result, it cannot
// hold the same name `hello` and we assume that a new file got created in the victim folder.
// That is enough to detect an archive breakout.
names, err := f.Readdirnames(2)
if err != nil {
// codepath taken if victim is not a folder
return fmt.Errorf("archive breakout: error reading directory content of %q: %v", victim, err)
}
for _, name := range names {
if name != "hello" {
// codepath taken if new file was created in victim folder
return fmt.Errorf("archive breakout: new file %q", name)
}
}
// Check victim/hello
f, err = os.Open(hello)
if err != nil {
// codepath taken if read permissions were removed
return fmt.Errorf("archive breakout: could not lstat %q: %v", hello, err)
}
defer f.Close()
b, err := ioutil.ReadAll(f)
if err != nil {
return err
}
fi, err := f.Stat()
if err != nil {
return err
}
if helloStat.IsDir() != fi.IsDir() ||
// TODO: cannot check for fi.ModTime() change
helloStat.Mode() != fi.Mode() ||
helloStat.Size() != fi.Size() ||
!bytes.Equal(helloData, b) {
// codepath taken if hello has been modified
return fmt.Errorf("archive breakout: file %q has been modified. Contents: expected=%q, got=%q. FileInfo: expected=%#v, got=%#v.", hello, helloData, b, helloStat, fi)
}
// Check that nothing in dest/ has the same content as victim/hello.
// Since victim/hello was generated with time.Now(), it is safe to assume
// that any file whose content matches exactly victim/hello, managed somehow
// to access victim/hello.
return filepath.Walk(dest, func(path string, info os.FileInfo, err error) error {
if info.IsDir() {
if err != nil {
// skip directory if error
return filepath.SkipDir
}
// enter directory
return nil
}
if err != nil {
// skip file if error
return nil
}
b, err := ioutil.ReadFile(path)
if err != nil {
// Houston, we have a problem. Aborting (space)walk.
return err
}
if bytes.Equal(helloData, b) {
return fmt.Errorf("archive breakout: file %q has been accessed via %q", hello, path)
}
return nil
})
}
package archive
import (
"archive/tar"
"bytes"
"io/ioutil"
)
// Generate generates a new archive from the content provided
// as input.
//
// `files` is a sequence of path/content pairs. A new file is
// added to the archive for each pair.
// If the last pair is incomplete, the file is created with an
// empty content. For example:
//
// Generate("foo.txt", "hello world", "emptyfile")
//
// The above call will return an archive with 2 files:
// * ./foo.txt with content "hello world"
// * ./empty with empty content
//
// FIXME: stream content instead of buffering
// FIXME: specify permissions and other archive metadata
func Generate(input ...string) (Archive, error) {
files := parseStringPairs(input...)
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
for _, file := range files {
name, content := file[0], file[1]
hdr := &tar.Header{
Name: name,
Size: int64(len(content)),
}
if err := tw.WriteHeader(hdr); err != nil {
return nil, err
}
if _, err := tw.Write([]byte(content)); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return ioutil.NopCloser(buf), nil
}
func parseStringPairs(input ...string) (output [][2]string) {
output = make([][2]string, 0, len(input)/2+1)
for i := 0; i < len(input); i += 2 {
var pair [2]string
pair[0] = input[i]
if i+1 < len(input) {
pair[1] = input[i+1]
}
output = append(output, pair)
}
return
}
package archive
import (
"archive/tar"
"bytes"
"io"
"testing"
)
func TestGenerateEmptyFile(t *testing.T) {
archive, err := Generate("emptyFile")
if err != nil {
t.Fatal(err)
}
if archive == nil {
t.Fatal("The generated archive should not be nil.")
}
expectedFiles := [][]string{
{"emptyFile", ""},
}
tr := tar.NewReader(archive)
actualFiles := make([][]string, 0, 10)
i := 0
for {
hdr, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
t.Fatal(err)
}
buf := new(bytes.Buffer)
buf.ReadFrom(tr)
content := buf.String()
actualFiles = append(actualFiles, []string{hdr.Name, content})
i++
}
if len(actualFiles) != len(expectedFiles) {
t.Fatalf("Number of expected file %d, got %d.", len(expectedFiles), len(actualFiles))
}
for i := 0; i < len(expectedFiles); i++ {
actual := actualFiles[i]
expected := expectedFiles[i]
if actual[0] != expected[0] {
t.Fatalf("Expected name '%s', Actual name '%s'", expected[0], actual[0])
}
if actual[1] != expected[1] {
t.Fatalf("Expected content '%s', Actual content '%s'", expected[1], actual[1])
}
}
}
func TestGenerateWithContent(t *testing.T) {
archive, err := Generate("file", "content")
if err != nil {
t.Fatal(err)
}
if archive == nil {
t.Fatal("The generated archive should not be nil.")
}
expectedFiles := [][]string{
{"file", "content"},
}
tr := tar.NewReader(archive)
actualFiles := make([][]string, 0, 10)
i := 0
for {
hdr, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
t.Fatal(err)
}
buf := new(bytes.Buffer)
buf.ReadFrom(tr)
content := buf.String()
actualFiles = append(actualFiles, []string{hdr.Name, content})
i++
}
if len(actualFiles) != len(expectedFiles) {
t.Fatalf("Number of expected file %d, got %d.", len(expectedFiles), len(actualFiles))
}
for i := 0; i < len(expectedFiles); i++ {
actual := actualFiles[i]
expected := expectedFiles[i]
if actual[0] != expected[0] {
t.Fatalf("Expected name '%s', Actual name '%s'", expected[0], actual[0])
}
if actual[1] != expected[1] {
t.Fatalf("Expected content '%s', Actual content '%s'", expected[1], actual[1])
}
}
}
package fileutils
import (
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/fsouza/go-dockerclient/vendor/github.com/Sirupsen/logrus"
)
func Exclusion(pattern string) bool {
return pattern[0] == '!'
}
func Empty(pattern string) bool {
return pattern == ""
}
// Cleanpatterns takes a slice of patterns returns a new
// slice of patterns cleaned with filepath.Clean, stripped
// of any empty patterns and lets the caller know whether the
// slice contains any exception patterns (prefixed with !).
func CleanPatterns(patterns []string) ([]string, [][]string, bool, error) {
// Loop over exclusion patterns and:
// 1. Clean them up.
// 2. Indicate whether we are dealing with any exception rules.
// 3. Error if we see a single exclusion marker on it's own (!).
cleanedPatterns := []string{}
patternDirs := [][]string{}
exceptions := false
for _, pattern := range patterns {
// Eliminate leading and trailing whitespace.
pattern = strings.TrimSpace(pattern)
if Empty(pattern) {
continue
}
if Exclusion(pattern) {
if len(pattern) == 1 {
logrus.Errorf("Illegal exclusion pattern: %s", pattern)
return nil, nil, false, errors.New("Illegal exclusion pattern: !")
}
exceptions = true
}
pattern = filepath.Clean(pattern)
cleanedPatterns = append(cleanedPatterns, pattern)
if Exclusion(pattern) {
pattern = pattern[1:]
}
patternDirs = append(patternDirs, strings.Split(pattern, "/"))
}
return cleanedPatterns, patternDirs, exceptions, nil
}
// Matches returns true if file matches any of the patterns
// and isn't excluded by any of the subsequent patterns.
func Matches(file string, patterns []string) (bool, error) {
file = filepath.Clean(file)
if file == "." {
// Don't let them exclude everything, kind of silly.
return false, nil
}
patterns, patDirs, _, err := CleanPatterns(patterns)
if err != nil {
return false, err
}
return OptimizedMatches(file, patterns, patDirs)
}
// Matches is basically the same as fileutils.Matches() but optimized for archive.go.
// It will assume that the inputs have been preprocessed and therefore the function
// doen't need to do as much error checking and clean-up. This was done to avoid
// repeating these steps on each file being checked during the archive process.
// The more generic fileutils.Matches() can't make these assumptions.
func OptimizedMatches(file string, patterns []string, patDirs [][]string) (bool, error) {
matched := false
parentPath := filepath.Dir(file)
parentPathDirs := strings.Split(parentPath, "/")
for i, pattern := range patterns {
negative := false
if Exclusion(pattern) {
negative = true
pattern = pattern[1:]
}
match, err := filepath.Match(pattern, file)
if err != nil {
logrus.Errorf("Error matching: %s (pattern: %s)", file, pattern)
return false, err
}
if !match && parentPath != "." {
// Check to see if the pattern matches one of our parent dirs.
if len(patDirs[i]) <= len(parentPathDirs) {
match, _ = filepath.Match(strings.Join(patDirs[i], "/"),
strings.Join(parentPathDirs[:len(patDirs[i])], "/"))
}
}
if match {
matched = !negative
}
}
if matched {
logrus.Debugf("Skipping excluded path: %s", file)
}
return matched, nil
}
func CopyFile(src, dst string) (int64, error) {
cleanSrc := filepath.Clean(src)
cleanDst := filepath.Clean(dst)
if cleanSrc == cleanDst {
return 0, nil
}
sf, err := os.Open(cleanSrc)
if err != nil {
return 0, err
}
defer sf.Close()
if err := os.Remove(cleanDst); err != nil && !os.IsNotExist(err) {
return 0, err
}
df, err := os.Create(cleanDst)
if err != nil {
return 0, err
}
defer df.Close()
return io.Copy(df, sf)
}
func GetTotalUsedFds() int {
if fds, err := ioutil.ReadDir(fmt.Sprintf("/proc/%d/fd", os.Getpid())); err != nil {
logrus.Errorf("Error opening /proc/%d/fd: %s", os.Getpid(), err)
} else {
return len(fds)
}
return -1
}
// ReadSymlinkedDirectory returns the target directory of a symlink.
// The target of the symbolic link may not be a file.
func ReadSymlinkedDirectory(path string) (string, error) {
var realPath string
var err error
if realPath, err = filepath.Abs(path); err != nil {
return "", fmt.Errorf("unable to get absolute path for %s: %s", path, err)
}
if realPath, err = filepath.EvalSymlinks(realPath); err != nil {
return "", fmt.Errorf("failed to canonicalise path for %s: %s", path, err)
}
realPathInfo, err := os.Stat(realPath)
if err != nil {
return "", fmt.Errorf("failed to stat target '%s' of '%s': %s", realPath, path, err)
}
if !realPathInfo.Mode().IsDir() {
return "", fmt.Errorf("canonical path points to a file '%s'", realPath)
}
return realPath, nil
}
package ioutils
import (
"bytes"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"io"
"math/big"
"sync"
"time"
)
type readCloserWrapper struct {
io.Reader
closer func() error
}
func (r *readCloserWrapper) Close() error {
return r.closer()
}
func NewReadCloserWrapper(r io.Reader, closer func() error) io.ReadCloser {
return &readCloserWrapper{
Reader: r,
closer: closer,
}
}
type readerErrWrapper struct {
reader io.Reader
closer func()
}
func (r *readerErrWrapper) Read(p []byte) (int, error) {
n, err := r.reader.Read(p)
if err != nil {
r.closer()
}
return n, err
}
func NewReaderErrWrapper(r io.Reader, closer func()) io.Reader {
return &readerErrWrapper{
reader: r,
closer: closer,
}
}
// bufReader allows the underlying reader to continue to produce
// output by pre-emptively reading from the wrapped reader.
// This is achieved by buffering this data in bufReader's
// expanding buffer.
type bufReader struct {
sync.Mutex
buf *bytes.Buffer
reader io.Reader
err error
wait sync.Cond
drainBuf []byte
reuseBuf []byte
maxReuse int64
resetTimeout time.Duration
bufLenResetThreshold int64
maxReadDataReset int64
}
func NewBufReader(r io.Reader) *bufReader {
var timeout int
if randVal, err := rand.Int(rand.Reader, big.NewInt(120)); err == nil {
timeout = int(randVal.Int64()) + 180
} else {
timeout = 300
}
reader := &bufReader{
buf: &bytes.Buffer{},
drainBuf: make([]byte, 1024),
reuseBuf: make([]byte, 4096),
maxReuse: 1000,
resetTimeout: time.Second * time.Duration(timeout),
bufLenResetThreshold: 100 * 1024,
maxReadDataReset: 10 * 1024 * 1024,
reader: r,
}
reader.wait.L = &reader.Mutex
go reader.drain()
return reader
}
func NewBufReaderWithDrainbufAndBuffer(r io.Reader, drainBuffer []byte, buffer *bytes.Buffer) *bufReader {
reader := &bufReader{
buf: buffer,
drainBuf: drainBuffer,
reader: r,
}
reader.wait.L = &reader.Mutex
go reader.drain()
return reader
}
func (r *bufReader) drain() {
var (
duration time.Duration
lastReset time.Time
now time.Time
reset bool
bufLen int64
dataSinceReset int64
maxBufLen int64
reuseBufLen int64
reuseCount int64
)
reuseBufLen = int64(len(r.reuseBuf))
lastReset = time.Now()
for {
n, err := r.reader.Read(r.drainBuf)
dataSinceReset += int64(n)
r.Lock()
bufLen = int64(r.buf.Len())
if bufLen > maxBufLen {
maxBufLen = bufLen
}
// Avoid unbounded growth of the buffer over time.
// This has been discovered to be the only non-intrusive
// solution to the unbounded growth of the buffer.
// Alternative solutions such as compression, multiple
// buffers, channels and other similar pieces of code
// were reducing throughput, overall Docker performance
// or simply crashed Docker.
// This solution releases the buffer when specific
// conditions are met to avoid the continuous resizing
// of the buffer for long lived containers.
//
// Move data to the front of the buffer if it's
// smaller than what reuseBuf can store
if bufLen > 0 && reuseBufLen >= bufLen {
n, _ := r.buf.Read(r.reuseBuf)
r.buf.Write(r.reuseBuf[0:n])
// Take action if the buffer has been reused too many
// times and if there's data in the buffer.
// The timeout is also used as means to avoid doing
// these operations more often or less often than
// required.
// The various conditions try to detect heavy activity
// in the buffer which might be indicators of heavy
// growth of the buffer.
} else if reuseCount >= r.maxReuse && bufLen > 0 {
now = time.Now()
duration = now.Sub(lastReset)
timeoutReached := duration >= r.resetTimeout
// The timeout has been reached and the
// buffered data couldn't be moved to the front
// of the buffer, so the buffer gets reset.
if timeoutReached && bufLen > reuseBufLen {
reset = true
}
// The amount of buffered data is too high now,
// reset the buffer.
if timeoutReached && maxBufLen >= r.bufLenResetThreshold {
reset = true
}
// Reset the buffer if a certain amount of
// data has gone through the buffer since the
// last reset.
if timeoutReached && dataSinceReset >= r.maxReadDataReset {
reset = true
}
// The buffered data is moved to a fresh buffer,
// swap the old buffer with the new one and
// reset all counters.
if reset {
newbuf := &bytes.Buffer{}
newbuf.ReadFrom(r.buf)
r.buf = newbuf
lastReset = now
reset = false
dataSinceReset = 0
maxBufLen = 0
reuseCount = 0
}
}
if err != nil {
r.err = err
} else {
r.buf.Write(r.drainBuf[0:n])
}
reuseCount++
r.wait.Signal()
r.Unlock()
if err != nil {
break
}
}
}
func (r *bufReader) Read(p []byte) (n int, err error) {
r.Lock()
defer r.Unlock()
for {
n, err = r.buf.Read(p)
if n > 0 {
return n, err
}
if r.err != nil {
return 0, r.err
}
r.wait.Wait()
}
}
func (r *bufReader) Close() error {
closer, ok := r.reader.(io.ReadCloser)
if !ok {
return nil
}
return closer.Close()
}
func HashData(src io.Reader) (string, error) {
h := sha256.New()
if _, err := io.Copy(h, src); err != nil {
return "", err
}
return "sha256:" + hex.EncodeToString(h.Sum(nil)), nil
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment