Commit d246c892 authored by Darren Shepherd's avatar Darren Shepherd

go mod vendor

parent 3ce8310a

Too many changes to show.

To preserve performance only 1000 of 1000+ files are displayed.

# Contributor Code of Conduct
As contributors and maintainers of this project,
and in the interest of fostering an open and welcoming community,
we pledge to respect all people who contribute through reporting issues,
posting feature requests, updating documentation,
submitting pull requests or patches, and other activities.
We are committed to making participation in this project
a harassment-free experience for everyone,
regardless of level of experience, gender, gender identity and expression,
sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, or nationality.
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing other's private information,
such as physical or electronic
addresses, without explicit permission
* Other unethical or unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct.
By adopting this Code of Conduct,
project maintainers commit themselves to fairly and consistently
applying these principles to every aspect of managing this project.
Project maintainers who do not follow or enforce the Code of Conduct
may be permanently removed from the project team.
This code of conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior
may be reported by opening an issue
or contacting one or more of the project maintainers.
This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.2.0,
available at [http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/)
# How to Create a New Release
## Prerequisites
Install [releasetool](https://github.com/googleapis/releasetool).
## Create a release
1. `cd` into the root directory, e.g., `~/go/src/cloud.google.com/go`
1. Checkout the master branch and ensure a clean and up-to-date state.
```
git checkout master
git pull --tags origin master
```
1. Run releasetool to generate a changelog from the last version. Note,
releasetool will prompt if the new version is a major, minor, or patch
version.
```
releasetool start --language go
```
1. Format the output to match CHANGES.md.
1. Submit a CL with the changes in CHANGES.md. The commit message should look
like this (where `v0.31.0` is instead the correct version number):
```
all: Release v0.31.0
```
1. Wait for approval from all reviewers and then submit the CL.
1. Return to the master branch and pull the release commit.
```
git checkout master
git pull origin master
```
1. Tag the current commit with the new version (e.g., `v0.31.0`)
```
releasetool tag --language go
```
1. Publish the tag to GoogleSource (i.e., origin):
```
git push origin $NEW_VERSION
```
1. Visit the [releases page][releases] on GitHub and click the "Draft a new
release" button. For tag version, enter the tag published in the previous
step. For the release title, use the version (e.g., `v0.31.0`). For the
description, copy the changes added to CHANGES.md.
[releases]: https://github.com/GoogleCloudPlatform/google-cloud-go/releases
(delete this for feature requests)
## Client
e.g. PubSub
## Describe Your Environment
e.g. Alpine Docker on GKE
## Expected Behavior
e.g. Messages arrive really fast.
## Actual Behavior
e.g. Messages arrive really slowly.
\ No newline at end of file
#!/bin/bash
# This script generates all GAPIC clients in this repo.
# One-time setup:
# cd path/to/googleapis # https://github.com/googleapis/googleapis
# virtualenv env
# . env/bin/activate
# pip install googleapis-artman
# deactivate
#
# Regenerate:
# cd path/to/googleapis
# . env/bin/activate
# $GOPATH/src/cloud.google.com/go/regen-gapic.sh
# deactivate
#
# Being in googleapis directory is important;
# that's where we find YAML files and where artman puts the "artman-genfiles" directory.
#
# NOTE: This script does not generate the "raw" gRPC client found in google.golang.org/genproto.
# To do that, use the regen.sh script in the genproto repo instead.
set -ex
APIS=(
google/api/expr/artman_cel.yaml
google/iam/artman_iam_admin.yaml
google/cloud/asset/artman_cloudasset_v1beta1.yaml
google/iam/credentials/artman_iamcredentials_v1.yaml
google/cloud/bigquery/datatransfer/artman_bigquerydatatransfer.yaml
google/cloud/dataproc/artman_dataproc_v1.yaml
google/cloud/dataproc/artman_dataproc_v1beta2.yaml
google/cloud/dialogflow/artman_dialogflow_v2.yaml
google/cloud/kms/artman_cloudkms.yaml
google/cloud/language/artman_language_v1.yaml
google/cloud/language/artman_language_v1beta2.yaml
google/cloud/oslogin/artman_oslogin_v1.yaml
google/cloud/oslogin/artman_oslogin_v1beta.yaml
google/cloud/redis/artman_redis_v1beta1.yaml
google/cloud/redis/artman_redis_v1.yaml
google/cloud/scheduler/artman_cloudscheduler_v1beta1.yaml
google/cloud/securitycenter/artman_securitycenter_v1beta1.yaml
google/cloud/speech/artman_speech_v1.yaml
google/cloud/speech/artman_speech_v1p1beta1.yaml
google/cloud/tasks/artman_cloudtasks_v2beta2.yaml
google/cloud/tasks/artman_cloudtasks_v2beta3.yaml
google/cloud/texttospeech/artman_texttospeech_v1.yaml
google/cloud/videointelligence/artman_videointelligence_v1.yaml
google/cloud/videointelligence/artman_videointelligence_v1beta1.yaml
google/cloud/videointelligence/artman_videointelligence_v1beta2.yaml
google/cloud/vision/artman_vision_v1.yaml
google/cloud/vision/artman_vision_v1p1beta1.yaml
google/devtools/artman_clouddebugger.yaml
google/devtools/clouderrorreporting/artman_errorreporting.yaml
google/devtools/cloudtrace/artman_cloudtrace_v1.yaml
google/devtools/cloudtrace/artman_cloudtrace_v2.yaml
google/devtools/containeranalysis/artman_containeranalysis_v1beta1.yaml
google/firestore/artman_firestore.yaml
google/logging/artman_logging.yaml
google/longrunning/artman_longrunning.yaml
google/monitoring/artman_monitoring.yaml
google/privacy/dlp/artman_dlp_v2.yaml
google/pubsub/artman_pubsub.yaml
google/spanner/admin/database/artman_spanner_admin_database.yaml
google/spanner/admin/instance/artman_spanner_admin_instance.yaml
google/spanner/artman_spanner.yaml
)
for api in "${APIS[@]}"; do
rm -rf artman-genfiles/*
artman --config "$api" generate go_gapic
cp -r artman-genfiles/gapi-*/cloud.google.com/go/* $GOPATH/src/cloud.google.com/go/
done
# NOTE(pongad): `sed -i` doesn't work on Macs, because -i option needs an argument.
# `-i ''` doesn't work on GNU, since the empty string is treated as a file name.
# So we just create the backup and delete it after.
ver=$(date +%Y%m%d)
find $GOPATH/src/cloud.google.com/go/ -name '*.go' -exec sed -i.backup -e "s/^const versionClient.*/const versionClient = \"$ver\"/" '{}' +
find $GOPATH/src/cloud.google.com/go/ -name '*.backup' -delete
#go list cloud.google.com/go/... | grep apiv | xargs go test
#go test -short cloud.google.com/go/...
#echo "googleapis version: $(git rev-parse HEAD)"
#!/bin/bash
# Selectively run tests for this repo, based on what has changed
# in a commit. Runs short tests for the whole repo, and full tests
# for changed directories.
set -e
prefix=cloud.google.com/go
dryrun=false
if [[ $1 == "-n" ]]; then
dryrun=true
shift
fi
if [[ $1 == "" ]]; then
echo >&2 "usage: $0 [-n] COMMIT"
exit 1
fi
# Files or directories that cause all tests to run if modified.
declare -A run_all
run_all=([.travis.yml]=1 [run-tests.sh]=1)
function run {
if $dryrun; then
echo $*
else
(set -x; $*)
fi
}
# Find all the packages that have changed in this commit.
declare -A changed_packages
for f in $(git diff-tree --no-commit-id --name-only -r $1); do
if [[ ${run_all[$f]} == 1 ]]; then
# This change requires a full test. Do it and exit.
run go test -race -v $prefix/...
exit
fi
# Map, e.g., "spanner/client.go" to "$prefix/spanner".
d=$(dirname $f)
if [[ $d == "." ]]; then
pkg=$prefix
else
pkg=$prefix/$d
fi
changed_packages[$pkg]=1
done
echo "changed packages: ${!changed_packages[*]}"
# Reports whether its argument, a package name, depends (recursively)
# on a changed package.
function depends_on_changed_package {
# According to go list, a package does not depend on itself, so
# we test that separately.
if [[ ${changed_packages[$1]} == 1 ]]; then
return 0
fi
for dep in $(go list -f '{{range .Deps}}{{.}} {{end}}' $1); do
if [[ ${changed_packages[$dep]} == 1 ]]; then
return 0
fi
done
return 1
}
# Collect the packages into two separate lists. (It is faster to call "go test" on a
# list of packages than to individually "go test" each one.)
shorts=
fulls=
for pkg in $(go list $prefix/...); do # for each package in the repo
if depends_on_changed_package $pkg; then # if it depends on a changed package
fulls="$fulls $pkg" # run the full test
else # otherwise
shorts="$shorts $pkg" # run the short test
fi
done
run go test -race -v -short $shorts
if [[ $fulls != "" ]]; then
run go test -race -v $fulls
fi
# How to Contribute
We'd love to accept your patches and contributions to this project. There are
just a few small guidelines you need to follow.
## Contributor License Agreement
Contributions to this project must be accompanied by a Contributor License
Agreement. You (or your employer) retain the copyright to your contribution;
this simply gives us permission to use and redistribute your contributions as
part of the project. Head over to <https://cla.developers.google.com/> to see
your current agreements on file or to sign a new one.
You generally only need to submit a CLA once, so if you've already submitted one
(even if it was for a different project), you probably don't need to do it
again.
## Code reviews
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.
## Community Guidelines
This project follows [Google's Open Source Community
Guidelines](https://opensource.google.com/conduct/).
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "master"
digest = "1:d3f4da757a2589770c6962aa3dc8edc8285889cdd12402dde001eafe9bb03c64"
name = "google.golang.org/api"
packages = [
"compute/v0.alpha",
"compute/v0.beta",
"compute/v1",
"gensupport",
"googleapi",
"googleapi/internal/uritemplates",
]
pruneopts = "UT"
revision = "583d854617af4d2080b5d2a24d72f7fc5a128ab2"
[[projects]]
digest = "1:e2999bf1bb6eddc2a6aa03fe5e6629120a53088926520ca3b4765f77d7ff7eab"
name = "k8s.io/klog"
packages = ["."]
pruneopts = "UT"
revision = "a5bc97fbc634d635061f3146511332c7e313a55a"
version = "v0.1.0"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"google.golang.org/api/compute/v0.alpha",
"google.golang.org/api/compute/v0.beta",
"google.golang.org/api/compute/v1",
"google.golang.org/api/googleapi",
"k8s.io/klog",
]
solver-name = "gps-cdcl"
solver-version = 1
# Gopkg.toml example
#
# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
#
# [prune]
# non-go = false
# go-tests = true
# unused-packages = true
[[constraint]]
branch = "master"
name = "google.golang.org/api"
[prune]
go-tests = true
unused-packages = true
# Copyright 2018 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
all: gen build test
.PHONY: gen
gen:
go run pkg/cloud/gen/main.go > pkg/cloud/gen.go
go run pkg/cloud/gen/main.go -mode test > pkg/cloud/gen_test.go
.PHONY: build
build: gen
go build ./...
mkdir -p bin
.PHONY: test
test:
go test ./...
# We cannot use golint currently due to errors in the GCP API naming.
# golint ./...
go vet ./...
.PHONY: clean
clean:
rm -rf ./bin
# k8s-cloud-provider
This repository contains support files for implementing the Kubernetes cloud
provider for the Google Cloud Platform. The code in this repository are the
Google Cloud specific portions of the cloud provider logic.
## Building
Run `make`.
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
# Perks for Go (golang.org)
Perks contains the Go package quantile that computes approximate quantiles over
an unbounded data stream within low memory and CPU bounds.
For more information and examples, see:
http://godoc.org/github.com/bmizerany/perks
A very special thank you and shout out to Graham Cormode (Rutgers University),
Flip Korn (AT&T Labs–Research), S. Muthukrishnan (Rutgers University), and
Divesh Srivastava (AT&T Labs–Research) for their research and publication of
[Effective Computation of Biased Quantiles over Data Streams](http://www.cs.rutgers.edu/~muthu/bquant.pdf)
Thank you, also:
* Armon Dadgar (@armon)
* Andrew Gerrand (@nf)
* Brad Fitzpatrick (@bradfitz)
* Keith Rarick (@kr)
FAQ:
Q: Why not move the quantile package into the project root?
A: I want to add more packages to perks later.
Copyright (C) 2013 Blake Mizerany
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
......@@ -77,15 +77,20 @@ func NewHighBiased(epsilon float64) *Stream {
// is guaranteed to be within (Quantile±Epsilon).
//
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
func NewTargeted(targets map[float64]float64) *Stream {
func NewTargeted(targetMap map[float64]float64) *Stream {
// Convert map to slice to avoid slow iterations on a map.
// ƒ is called on the hot path, so converting the map to a slice
// beforehand results in significant CPU savings.
targets := targetMapToSlice(targetMap)
ƒ := func(s *stream, r float64) float64 {
var m = math.MaxFloat64
var f float64
for quantile, epsilon := range targets {
if quantile*s.n <= r {
f = (2 * epsilon * r) / quantile
for _, t := range targets {
if t.quantile*s.n <= r {
f = (2 * t.epsilon * r) / t.quantile
} else {
f = (2 * epsilon * (s.n - r)) / (1 - quantile)
f = (2 * t.epsilon * (s.n - r)) / (1 - t.quantile)
}
if f < m {
m = f
......@@ -96,6 +101,25 @@ func NewTargeted(targets map[float64]float64) *Stream {
return newStream(ƒ)
}
type target struct {
quantile float64
epsilon float64
}
func targetMapToSlice(targetMap map[float64]float64) []target {
targets := make([]target, 0, len(targetMap))
for quantile, epsilon := range targetMap {
t := target{
quantile: quantile,
epsilon: epsilon,
}
targets = append(targets, t)
}
return targets
}
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
// design. Take care when using across multiple goroutines.
type Stream struct {
......
Copyright (c) 2017, Gorillalabs
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
http://www.opensource.org/licenses/MIT
# go-powershell
This package is inspired by [jPowerShell](https://github.com/profesorfalken/jPowerShell)
and allows one to run and remote-control a PowerShell session. Use this if you
don't have a static script that you want to execute, bur rather run dynamic
commands.
## Installation
go get github.com/bhendo/go-powershell
## Usage
To start a PowerShell shell, you need a backend. Backends take care of starting
and controlling the actual powershell.exe process. In most cases, you will want
to use the Local backend, which just uses ``os/exec`` to start the process.
```go
package main
import (
"fmt"
ps "github.com/bhendo/go-powershell"
"github.com/bhendo/go-powershell/backend"
)
func main() {
// choose a backend
back := &backend.Local{}
// start a local powershell process
shell, err := ps.New(back)
if err != nil {
panic(err)
}
defer shell.Exit()
// ... and interact with it
stdout, stderr, err := shell.Execute("Get-WmiObject -Class Win32_Processor")
if err != nil {
panic(err)
}
fmt.Println(stdout)
}
```
## Remote Sessions
You can use an existing PS shell to use PSSession cmdlets to connect to remote
computers. Instead of manually handling that, you can use the Session middleware,
which takes care of authentication. Note that you can still use the "raw" shell
to execute commands on the computer where the powershell host process is running.
```go
package main
import (
"fmt"
ps "github.com/bhendo/go-powershell"
"github.com/bhendo/go-powershell/backend"
"github.com/bhendo/go-powershell/middleware"
)
func main() {
// choose a backend
back := &backend.Local{}
// start a local powershell process
shell, err := ps.New(back)
if err != nil {
panic(err)
}
// prepare remote session configuration
config := middleware.NewSessionConfig()
config.ComputerName = "remote-pc-1"
// create a new shell by wrapping the existing one in the session middleware
session, err := middleware.NewSession(shell, config)
if err != nil {
panic(err)
}
defer session.Exit() // will also close the underlying ps shell!
// everything run via the session is run on the remote machine
stdout, stderr, err = session.Execute("Get-WmiObject -Class Win32_Processor")
if err != nil {
panic(err)
}
fmt.Println(stdout)
}
```
Note that a single shell instance is not safe for concurrent use, as are remote
sessions. You can have as many remote sessions using the same shell as you like,
but you must execute commands serially. If you need concurrency, you can just
spawn multiple PowerShell processes (i.e. call ``.New()`` multiple times).
Also, note that all commands that you execute are wrapped in special echo
statements to delimit the stdout/stderr streams. After ``.Execute()``ing a command,
you can therefore not access ``$LastExitCode`` anymore and expect meaningful
results.
## License
MIT, see LICENSE file.
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package backend
import (
"io"
"os/exec"
"github.com/juju/errors"
)
type Local struct{}
func (b *Local) StartProcess(cmd string, args ...string) (Waiter, io.Writer, io.Reader, io.Reader, error) {
command := exec.Command(cmd, args...)
stdin, err := command.StdinPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the PowerShell's stdin stream")
}
stdout, err := command.StdoutPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the PowerShell's stdout stream")
}
stderr, err := command.StderrPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the PowerShell's stderr stream")
}
err = command.Start()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not spawn PowerShell process")
}
return command, stdin, stdout, stderr, nil
}
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package backend
import (
"fmt"
"io"
"regexp"
"strings"
"github.com/juju/errors"
)
// sshSession exists so we don't create a hard dependency on crypto/ssh.
type sshSession interface {
Waiter
StdinPipe() (io.WriteCloser, error)
StdoutPipe() (io.Reader, error)
StderrPipe() (io.Reader, error)
Start(string) error
}
type SSH struct {
Session sshSession
}
func (b *SSH) StartProcess(cmd string, args ...string) (Waiter, io.Writer, io.Reader, io.Reader, error) {
stdin, err := b.Session.StdinPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the SSH session's stdin stream")
}
stdout, err := b.Session.StdoutPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the SSH session's stdout stream")
}
stderr, err := b.Session.StderrPipe()
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not get hold of the SSH session's stderr stream")
}
err = b.Session.Start(b.createCmd(cmd, args))
if err != nil {
return nil, nil, nil, nil, errors.Annotate(err, "Could not spawn process via SSH")
}
return b.Session, stdin, stdout, stderr, nil
}
func (b *SSH) createCmd(cmd string, args []string) string {
parts := []string{cmd}
simple := regexp.MustCompile(`^[a-z0-9_/.~+-]+$`)
for _, arg := range args {
if !simple.MatchString(arg) {
arg = b.quote(arg)
}
parts = append(parts, arg)
}
return strings.Join(parts, " ")
}
func (b *SSH) quote(s string) string {
return fmt.Sprintf(`"%s"`, s)
}
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package backend
import "io"
type Waiter interface {
Wait() error
}
type Starter interface {
StartProcess(cmd string, args ...string) (Waiter, io.Writer, io.Reader, io.Reader, error)
}
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package powershell
import (
"fmt"
"io"
"strings"
"sync"
"github.com/bhendo/go-powershell/backend"
"github.com/bhendo/go-powershell/utils"
"github.com/juju/errors"
)
const newline = "\r\n"
type Shell interface {
Execute(cmd string) (string, string, error)
Exit()
}
type shell struct {
handle backend.Waiter
stdin io.Writer
stdout io.Reader
stderr io.Reader
}
func New(backend backend.Starter) (Shell, error) {
handle, stdin, stdout, stderr, err := backend.StartProcess("powershell.exe", "-NoExit", "-Command", "-")
if err != nil {
return nil, err
}
return &shell{handle, stdin, stdout, stderr}, nil
}
func (s *shell) Execute(cmd string) (string, string, error) {
if s.handle == nil {
return "", "", errors.Annotate(errors.New(cmd), "Cannot execute commands on closed shells.")
}
outBoundary := createBoundary()
errBoundary := createBoundary()
// wrap the command in special markers so we know when to stop reading from the pipes
full := fmt.Sprintf("%s; echo '%s'; [Console]::Error.WriteLine('%s')%s", cmd, outBoundary, errBoundary, newline)
_, err := s.stdin.Write([]byte(full))
if err != nil {
return "", "", errors.Annotate(errors.Annotate(err, cmd), "Could not send PowerShell command")
}
// read stdout and stderr
sout := ""
serr := ""
waiter := &sync.WaitGroup{}
waiter.Add(2)
go streamReader(s.stdout, outBoundary, &sout, waiter)
go streamReader(s.stderr, errBoundary, &serr, waiter)
waiter.Wait()
if len(serr) > 0 {
return sout, serr, errors.Annotate(errors.New(cmd), serr)
}
return sout, serr, nil
}
func (s *shell) Exit() {
s.stdin.Write([]byte("exit" + newline))
// if it's possible to close stdin, do so (some backends, like the local one,
// do support it)
closer, ok := s.stdin.(io.Closer)
if ok {
closer.Close()
}
s.handle.Wait()
s.handle = nil
s.stdin = nil
s.stdout = nil
s.stderr = nil
}
func streamReader(stream io.Reader, boundary string, buffer *string, signal *sync.WaitGroup) error {
// read all output until we have found our boundary token
output := ""
bufsize := 64
marker := boundary + newline
for {
buf := make([]byte, bufsize)
read, err := stream.Read(buf)
if err != nil {
return err
}
output = output + string(buf[:read])
if strings.HasSuffix(output, marker) {
break
}
}
*buffer = strings.TrimSuffix(output, marker)
signal.Done()
return nil
}
func createBoundary() string {
return "$gorilla" + utils.CreateRandomString(12) + "$"
}
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package utils
import "strings"
func QuoteArg(s string) string {
return "'" + strings.Replace(s, "'", "\"", -1) + "'"
}
// Copyright (c) 2017 Gorillalabs. All rights reserved.
package utils
import (
"crypto/rand"
"encoding/hex"
)
func CreateRandomString(bytes int) string {
c := bytes
b := make([]byte, c)
_, err := rand.Read(b)
if err != nil {
panic(err)
}
return hex.EncodeToString(b)
}
{
"author": "blang",
"bugs": {
"URL": "https://github.com/blang/semver/issues",
"url": "https://github.com/blang/semver/issues"
},
"gx": {
"dvcsimport": "github.com/blang/semver"
},
"gxVersion": "0.10.0",
"language": "go",
"license": "MIT",
"name": "semver",
"releaseCmd": "git commit -a -m \"gx publish $VERSION\"",
"version": "3.4.0"
}
......@@ -2,10 +2,33 @@ package semver
import (
"fmt"
"strconv"
"strings"
"unicode"
)
type wildcardType int
const (
noneWildcard wildcardType = iota
majorWildcard wildcardType = 1
minorWildcard wildcardType = 2
patchWildcard wildcardType = 3
)
func wildcardTypefromInt(i int) wildcardType {
switch i {
case 1:
return majorWildcard
case 2:
return minorWildcard
case 3:
return patchWildcard
default:
return noneWildcard
}
}
type comparator func(Version, Version) bool
var (
......@@ -92,8 +115,12 @@ func ParseRange(s string) (Range, error) {
if err != nil {
return nil, err
}
expandedParts, err := expandWildcardVersion(orParts)
if err != nil {
return nil, err
}
var orFn Range
for _, p := range orParts {
for _, p := range expandedParts {
var andFn Range
for _, ap := range p {
opStr, vStr, err := splitComparatorVersion(ap)
......@@ -164,20 +191,39 @@ func buildVersionRange(opStr, vStr string) (*versionRange, error) {
}
// splitAndTrim splits a range string by spaces and cleans leading and trailing spaces
// inArray checks if a byte is contained in an array of bytes
func inArray(s byte, list []byte) bool {
for _, el := range list {
if el == s {
return true
}
}
return false
}
// splitAndTrim splits a range string by spaces and cleans whitespaces
func splitAndTrim(s string) (result []string) {
last := 0
var lastChar byte
excludeFromSplit := []byte{'>', '<', '='}
for i := 0; i < len(s); i++ {
if s[i] == ' ' {
if s[i] == ' ' && !inArray(lastChar, excludeFromSplit) {
if last < i-1 {
result = append(result, s[last:i])
}
last = i + 1
} else if s[i] != ' ' {
lastChar = s[i]
}
}
if last < len(s)-1 {
result = append(result, s[last:])
}
for i, v := range result {
result[i] = strings.Replace(v, " ", "", -1)
}
// parts := strings.Split(s, " ")
// for _, x := range parts {
// if s := strings.TrimSpace(x); len(s) != 0 {
......@@ -188,7 +234,6 @@ func splitAndTrim(s string) (result []string) {
}
// splitComparatorVersion splits the comparator from the version.
// Spaces between the comparator and the version are not allowed.
// Input must be free of leading or trailing spaces.
func splitComparatorVersion(s string) (string, string, error) {
i := strings.IndexFunc(s, unicode.IsDigit)
......@@ -198,6 +243,144 @@ func splitComparatorVersion(s string) (string, string, error) {
return strings.TrimSpace(s[0:i]), s[i:], nil
}
// getWildcardType will return the type of wildcard that the
// passed version contains
func getWildcardType(vStr string) wildcardType {
parts := strings.Split(vStr, ".")
nparts := len(parts)
wildcard := parts[nparts-1]
possibleWildcardType := wildcardTypefromInt(nparts)
if wildcard == "x" {
return possibleWildcardType
}
return noneWildcard
}
// createVersionFromWildcard will convert a wildcard version
// into a regular version, replacing 'x's with '0's, handling
// special cases like '1.x.x' and '1.x'
func createVersionFromWildcard(vStr string) string {
// handle 1.x.x
vStr2 := strings.Replace(vStr, ".x.x", ".x", 1)
vStr2 = strings.Replace(vStr2, ".x", ".0", 1)
parts := strings.Split(vStr2, ".")
// handle 1.x
if len(parts) == 2 {
return vStr2 + ".0"
}
return vStr2
}
// incrementMajorVersion will increment the major version
// of the passed version
func incrementMajorVersion(vStr string) (string, error) {
parts := strings.Split(vStr, ".")
i, err := strconv.Atoi(parts[0])
if err != nil {
return "", err
}
parts[0] = strconv.Itoa(i + 1)
return strings.Join(parts, "."), nil
}
// incrementMajorVersion will increment the minor version
// of the passed version
func incrementMinorVersion(vStr string) (string, error) {
parts := strings.Split(vStr, ".")
i, err := strconv.Atoi(parts[1])
if err != nil {
return "", err
}
parts[1] = strconv.Itoa(i + 1)
return strings.Join(parts, "."), nil
}
// expandWildcardVersion will expand wildcards inside versions
// following these rules:
//
// * when dealing with patch wildcards:
// >= 1.2.x will become >= 1.2.0
// <= 1.2.x will become < 1.3.0
// > 1.2.x will become >= 1.3.0
// < 1.2.x will become < 1.2.0
// != 1.2.x will become < 1.2.0 >= 1.3.0
//
// * when dealing with minor wildcards:
// >= 1.x will become >= 1.0.0
// <= 1.x will become < 2.0.0
// > 1.x will become >= 2.0.0
// < 1.0 will become < 1.0.0
// != 1.x will become < 1.0.0 >= 2.0.0
//
// * when dealing with wildcards without
// version operator:
// 1.2.x will become >= 1.2.0 < 1.3.0
// 1.x will become >= 1.0.0 < 2.0.0
func expandWildcardVersion(parts [][]string) ([][]string, error) {
var expandedParts [][]string
for _, p := range parts {
var newParts []string
for _, ap := range p {
if strings.Index(ap, "x") != -1 {
opStr, vStr, err := splitComparatorVersion(ap)
if err != nil {
return nil, err
}
versionWildcardType := getWildcardType(vStr)
flatVersion := createVersionFromWildcard(vStr)
var resultOperator string
var shouldIncrementVersion bool
switch opStr {
case ">":
resultOperator = ">="
shouldIncrementVersion = true
case ">=":
resultOperator = ">="
case "<":
resultOperator = "<"
case "<=":
resultOperator = "<"
shouldIncrementVersion = true
case "", "=", "==":
newParts = append(newParts, ">="+flatVersion)
resultOperator = "<"
shouldIncrementVersion = true
case "!=", "!":
newParts = append(newParts, "<"+flatVersion)
resultOperator = ">="
shouldIncrementVersion = true
}
var resultVersion string
if shouldIncrementVersion {
switch versionWildcardType {
case patchWildcard:
resultVersion, _ = incrementMinorVersion(flatVersion)
case minorWildcard:
resultVersion, _ = incrementMajorVersion(flatVersion)
}
} else {
resultVersion = flatVersion
}
ap = resultOperator + resultVersion
}
newParts = append(newParts, ap)
}
expandedParts = append(expandedParts, newParts)
}
return expandedParts, nil
}
func parseComparator(s string) comparator {
switch s {
case "==":
......@@ -222,3 +405,12 @@ func parseComparator(s string) comparator {
return nil
}
// MustParseRange is like ParseRange but panics if the range cannot be parsed.
func MustParseRange(s string) Range {
r, err := ParseRange(s)
if err != nil {
panic(`semver: ParseRange(` + s + `): ` + err.Error())
}
return r
}
......@@ -200,6 +200,29 @@ func Make(s string) (Version, error) {
return Parse(s)
}
// ParseTolerant allows for certain version specifications that do not strictly adhere to semver
// specs to be parsed by this library. It does so by normalizing versions before passing them to
// Parse(). It currently trims spaces, removes a "v" prefix, and adds a 0 patch number to versions
// with only major and minor components specified
func ParseTolerant(s string) (Version, error) {
s = strings.TrimSpace(s)
s = strings.TrimPrefix(s, "v")
// Split into major.minor.(patch+pr+meta)
parts := strings.SplitN(s, ".", 3)
if len(parts) < 3 {
if strings.ContainsAny(parts[len(parts)-1], "+-") {
return Version{}, errors.New("Short version cannot contain PreRelease/Build meta data")
}
for len(parts) < 3 {
parts = append(parts, "0")
}
s = strings.Join(parts, ".")
}
return Parse(s)
}
// Parse parses version string and returns a validated Version or error
func Parse(s string) (Version, error) {
if len(s) == 0 {
......
*.test
*.out
*.mprof
vendor/github.com/buger/goterm/
prof.cpu
prof.mem
language: go
go:
- 1.7.x
- 1.8.x
- 1.9.x
- 1.10.x
- 1.11.x
script: go test -v ./.
FROM golang:1.6
RUN go get github.com/Jeffail/gabs
RUN go get github.com/bitly/go-simplejson
RUN go get github.com/pquerna/ffjson
RUN go get github.com/antonholmquist/jason
RUN go get github.com/mreiferson/go-ujson
RUN go get -tags=unsafe -u github.com/ugorji/go/codec
RUN go get github.com/mailru/easyjson
WORKDIR /go/src/github.com/buger/jsonparser
ADD . /go/src/github.com/buger/jsonparser
\ No newline at end of file
MIT License
Copyright (c) 2016 Leonid Bugaev
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOURCE = parser.go
CONTAINER = jsonparser
SOURCE_PATH = /go/src/github.com/buger/jsonparser
BENCHMARK = JsonParser
BENCHTIME = 5s
TEST = .
DRUN = docker run -v `pwd`:$(SOURCE_PATH) -i -t $(CONTAINER)
build:
docker build -t $(CONTAINER) .
race:
$(DRUN) --env GORACE="halt_on_error=1" go test ./. $(ARGS) -v -race -timeout 15s
bench:
$(DRUN) go test $(LDFLAGS) -test.benchmem -bench $(BENCHMARK) ./benchmark/ $(ARGS) -benchtime $(BENCHTIME) -v
bench_local:
$(DRUN) go test $(LDFLAGS) -test.benchmem -bench . $(ARGS) -benchtime $(BENCHTIME) -v
profile:
$(DRUN) go test $(LDFLAGS) -test.benchmem -bench $(BENCHMARK) ./benchmark/ $(ARGS) -memprofile mem.mprof -v
$(DRUN) go test $(LDFLAGS) -test.benchmem -bench $(BENCHMARK) ./benchmark/ $(ARGS) -cpuprofile cpu.out -v
$(DRUN) go test $(LDFLAGS) -test.benchmem -bench $(BENCHMARK) ./benchmark/ $(ARGS) -c
test:
$(DRUN) go test $(LDFLAGS) ./ -run $(TEST) -timeout 10s $(ARGS) -v
fmt:
$(DRUN) go fmt ./...
vet:
$(DRUN) go vet ./.
bash:
$(DRUN) /bin/bash
\ No newline at end of file
package jsonparser
import (
bio "bytes"
)
// minInt64 '-9223372036854775808' is the smallest representable number in int64
const minInt64 = `9223372036854775808`
// About 2x faster then strconv.ParseInt because it only supports base 10, which is enough for JSON
func parseInt(bytes []byte) (v int64, ok bool, overflow bool) {
if len(bytes) == 0 {
return 0, false, false
}
var neg bool = false
if bytes[0] == '-' {
neg = true
bytes = bytes[1:]
}
var b int64 = 0
for _, c := range bytes {
if c >= '0' && c <= '9' {
b = (10 * v) + int64(c-'0')
} else {
return 0, false, false
}
if overflow = (b < v); overflow {
break
}
v = b
}
if overflow {
if neg && bio.Equal(bytes, []byte(minInt64)) {
return b, true, false
}
return 0, false, true
}
if neg {
return -v, true, false
} else {
return v, true, false
}
}
// +build appengine appenginevm
package jsonparser
import (
"strconv"
)
// See fastbytes_unsafe.go for explanation on why *[]byte is used (signatures must be consistent with those in that file)
func equalStr(b *[]byte, s string) bool {
return string(*b) == s
}
func parseFloat(b *[]byte) (float64, error) {
return strconv.ParseFloat(string(*b), 64)
}
func bytesToString(b *[]byte) string {
return string(*b)
}
func StringToBytes(s string) []byte {
return []byte(s)
}
// +build !appengine,!appenginevm
package jsonparser
import (
"reflect"
"strconv"
"unsafe"
)
//
// The reason for using *[]byte rather than []byte in parameters is an optimization. As of Go 1.6,
// the compiler cannot perfectly inline the function when using a non-pointer slice. That is,
// the non-pointer []byte parameter version is slower than if its function body is manually
// inlined, whereas the pointer []byte version is equally fast to the manually inlined
// version. Instruction count in assembly taken from "go tool compile" confirms this difference.
//
// TODO: Remove hack after Go 1.7 release
//
func equalStr(b *[]byte, s string) bool {
return *(*string)(unsafe.Pointer(b)) == s
}
func parseFloat(b *[]byte) (float64, error) {
return strconv.ParseFloat(*(*string)(unsafe.Pointer(b)), 64)
}
// A hack until issue golang/go#2632 is fixed.
// See: https://github.com/golang/go/issues/2632
func bytesToString(b *[]byte) string {
return *(*string)(unsafe.Pointer(b))
}
func StringToBytes(s string) []byte {
sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
bh := reflect.SliceHeader{
Data: sh.Data,
Len: sh.Len,
Cap: sh.Len,
}
return *(*[]byte)(unsafe.Pointer(&bh))
}
package jsonparser
import (
"bytes"
"unicode/utf8"
)
// JSON Unicode stuff: see https://tools.ietf.org/html/rfc7159#section-7
const supplementalPlanesOffset = 0x10000
const highSurrogateOffset = 0xD800
const lowSurrogateOffset = 0xDC00
const basicMultilingualPlaneReservedOffset = 0xDFFF
const basicMultilingualPlaneOffset = 0xFFFF
func combineUTF16Surrogates(high, low rune) rune {
return supplementalPlanesOffset + (high-highSurrogateOffset)<<10 + (low - lowSurrogateOffset)
}
const badHex = -1
func h2I(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'A' && c <= 'F':
return int(c - 'A' + 10)
case c >= 'a' && c <= 'f':
return int(c - 'a' + 10)
}
return badHex
}
// decodeSingleUnicodeEscape decodes a single \uXXXX escape sequence. The prefix \u is assumed to be present and
// is not checked.
// In JSON, these escapes can either come alone or as part of "UTF16 surrogate pairs" that must be handled together.
// This function only handles one; decodeUnicodeEscape handles this more complex case.
func decodeSingleUnicodeEscape(in []byte) (rune, bool) {
// We need at least 6 characters total
if len(in) < 6 {
return utf8.RuneError, false
}
// Convert hex to decimal
h1, h2, h3, h4 := h2I(in[2]), h2I(in[3]), h2I(in[4]), h2I(in[5])
if h1 == badHex || h2 == badHex || h3 == badHex || h4 == badHex {
return utf8.RuneError, false
}
// Compose the hex digits
return rune(h1<<12 + h2<<8 + h3<<4 + h4), true
}
// isUTF16EncodedRune checks if a rune is in the range for non-BMP characters,
// which is used to describe UTF16 chars.
// Source: https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
func isUTF16EncodedRune(r rune) bool {
return highSurrogateOffset <= r && r <= basicMultilingualPlaneReservedOffset
}
func decodeUnicodeEscape(in []byte) (rune, int) {
if r, ok := decodeSingleUnicodeEscape(in); !ok {
// Invalid Unicode escape
return utf8.RuneError, -1
} else if r <= basicMultilingualPlaneOffset && !isUTF16EncodedRune(r) {
// Valid Unicode escape in Basic Multilingual Plane
return r, 6
} else if r2, ok := decodeSingleUnicodeEscape(in[6:]); !ok { // Note: previous decodeSingleUnicodeEscape success guarantees at least 6 bytes remain
// UTF16 "high surrogate" without manditory valid following Unicode escape for the "low surrogate"
return utf8.RuneError, -1
} else if r2 < lowSurrogateOffset {
// Invalid UTF16 "low surrogate"
return utf8.RuneError, -1
} else {
// Valid UTF16 surrogate pair
return combineUTF16Surrogates(r, r2), 12
}
}
// backslashCharEscapeTable: when '\X' is found for some byte X, it is to be replaced with backslashCharEscapeTable[X]
var backslashCharEscapeTable = [...]byte{
'"': '"',
'\\': '\\',
'/': '/',
'b': '\b',
'f': '\f',
'n': '\n',
'r': '\r',
't': '\t',
}
// unescapeToUTF8 unescapes the single escape sequence starting at 'in' into 'out' and returns
// how many characters were consumed from 'in' and emitted into 'out'.
// If a valid escape sequence does not appear as a prefix of 'in', (-1, -1) to signal the error.
func unescapeToUTF8(in, out []byte) (inLen int, outLen int) {
if len(in) < 2 || in[0] != '\\' {
// Invalid escape due to insufficient characters for any escape or no initial backslash
return -1, -1
}
// https://tools.ietf.org/html/rfc7159#section-7
switch e := in[1]; e {
case '"', '\\', '/', 'b', 'f', 'n', 'r', 't':
// Valid basic 2-character escapes (use lookup table)
out[0] = backslashCharEscapeTable[e]
return 2, 1
case 'u':
// Unicode escape
if r, inLen := decodeUnicodeEscape(in); inLen == -1 {
// Invalid Unicode escape
return -1, -1
} else {
// Valid Unicode escape; re-encode as UTF8
outLen := utf8.EncodeRune(out, r)
return inLen, outLen
}
}
return -1, -1
}
// unescape unescapes the string contained in 'in' and returns it as a slice.
// If 'in' contains no escaped characters:
// Returns 'in'.
// Else, if 'out' is of sufficient capacity (guaranteed if cap(out) >= len(in)):
// 'out' is used to build the unescaped string and is returned with no extra allocation
// Else:
// A new slice is allocated and returned.
func Unescape(in, out []byte) ([]byte, error) {
firstBackslash := bytes.IndexByte(in, '\\')
if firstBackslash == -1 {
return in, nil
}
// Get a buffer of sufficient size (allocate if needed)
if cap(out) < len(in) {
out = make([]byte, len(in))
} else {
out = out[0:len(in)]
}
// Copy the first sequence of unescaped bytes to the output and obtain a buffer pointer (subslice)
copy(out, in[:firstBackslash])
in = in[firstBackslash:]
buf := out[firstBackslash:]
for len(in) > 0 {
// Unescape the next escaped character
inLen, bufLen := unescapeToUTF8(in, buf)
if inLen == -1 {
return nil, MalformedStringEscapeError
}
in = in[inLen:]
buf = buf[bufLen:]
// Copy everything up until the next backslash
nextBackslash := bytes.IndexByte(in, '\\')
if nextBackslash == -1 {
copy(buf, in)
buf = buf[len(in):]
break
} else {
copy(buf, in[:nextBackslash])
buf = buf[nextBackslash:]
in = in[nextBackslash:]
}
}
// Trim the out buffer to the amount that was actually emitted
return out[:len(out)-len(buf)], nil
}
language: go
go:
- 1.3
- 1.4
- 1.5
- tip
gettext-go
==========
PkgDoc: [http://godoc.org/github.com/chai2010/gettext-go/gettext](http://godoc.org/github.com/chai2010/gettext-go/gettext)
Install
========
1. `go get github.com/chai2010/gettext-go/gettext`
2. `go run hello.go`
The godoc.org or gowalker.org has more information.
Example
=======
```Go
package main
import (
"fmt"
"github.com/chai2010/gettext-go/gettext"
)
func main() {
gettext.SetLocale("zh_CN")
gettext.Textdomain("hello")
gettext.BindTextdomain("hello", "local", nil)
// gettext.BindTextdomain("hello", "local", nil) // from local dir
// gettext.BindTextdomain("hello", "local.zip", nil) // from local zip file
// gettext.BindTextdomain("hello", "local.zip", zipData) // from embedded zip data
// translate source text
fmt.Println(gettext.Gettext("Hello, world!"))
// Output: 你好, 世界!
// if no msgctxt in PO file (only msgid and msgstr),
// specify context as "" by
fmt.Println(gettext.PGettext("", "Hello, world!"))
// Output: 你好, 世界!
// translate resource
fmt.Println(string(gettext.Getdata("poems.txt"))))
// Output: ...
}
```
Go file: [hello.go](https://github.com/chai2010/gettext-go/blob/master/examples/hello.go); PO file: [hello.po](https://github.com/chai2010/gettext-go/blob/master/examples/local/default/LC_MESSAGES/hello.po);
BUGS
====
Please report bugs to <chaishushan@gmail.com>.
Thanks!
dist/*
cli/serve/static.rice-box.go
.coverprofile
coverprofile.txt
gopath
sudo: false
language: go
go:
- 1.8.x
- 1.9.x
- 1.10.x
- master
# Install g++-4.8 to support std=c++11 for github.com/google/certificate-transparency/go/merkletree
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-4.8
install:
- if [ "$CXX" = "g++" ]; then export CXX="g++-4.8"; fi
# Used by the certdb tests
services:
- mysql
- postgresql
before_install:
# CFSSL consists of multiple Go packages, which refer to each other by
# their absolute GitHub path, e.g. github.com/cloudflare/crypto/pkcs7.
# That means, by default, if someone forks the repo and makes changes across
# multiple packages within CFSSL, Travis won't pass for the branch on their
# own repo. To fix that, we move the directory
- mkdir -p $TRAVIS_BUILD_DIR $GOPATH/src/github.com/cloudflare
- test ! -d $GOPATH/src/github.com/cloudflare/cfssl && mv $TRAVIS_BUILD_DIR $GOPATH/src/github.com/cloudflare/cfssl || true
# Only build pull requests, pushes to the master branch, and branches
# starting with `test-`. This is a convenient way to push branches to
# your own fork of the repostiory to ensure Travis passes before submitting
# a PR. For instance, you might run:
# git push myremote branchname:test-branchname
branches:
only:
- master
- /^test-.*$/
before_script:
- go get -u github.com/golang/lint/golint
- go get -v github.com/GeertJohan/fgt
# Setup DBs + run migrations
- go get bitbucket.org/liamstask/goose/cmd/goose
- if [[ $(uname -s) == 'Linux' ]]; then
psql -c 'create database certdb_development;' -U postgres;
goose -path $GOPATH/src/github.com/cloudflare/cfssl/certdb/pg up;
mysql -e 'create database certdb_development;' -u root;
goose -path $GOPATH/src/github.com/cloudflare/cfssl/certdb/mysql up;
fi
script:
- ./test.sh
notifications:
email:
recipients:
- cbroglie@cloudflare.com
- kyle@cloudflare.com
- nick@cloudflare.com
on_success: never
on_failure: change
env:
global:
- secure: "OmaaZ3jhU9VQ/0SYpenUJEfnmKy/MwExkefFRpDbkRSu/hTQpxxALAZV5WEHo7gxLRMRI0pytLo7w+lAd2FlX1CNcyY62MUicta/8P2twsxp+lR3v1bJ7dwk6qsDbO7Nvv3BKPCDQCHUkggbAEJaHEQGdLk4ursNEB1aGimuCEc="
- GO15VENDOREXPERIMENT=1
matrix:
- BUILD_TAGS="postgresql mysql"
matrix:
include:
- os: osx
go: 1.8.1
env: BUILD_TAGS=
after_success:
- bash <(curl -s https://codecov.io/bash) -f coverprofile.txt
# How to Build CFSSL
## Docker
The requirements to build `CFSSL` are:
1. A running instance of Docker
2. The `bash` shell
To build, run:
$ script/build-docker
This is will build by default all the cfssl command line utilities
for darwin (OSX), linux, and windows for i386 and amd64 and output the
binaries in the current path.
To build a specific platform and OS, run:
$ script/build-docker -os="darwin" -arch="amd64"
Note: for cross-compilation compatibility, the Docker build process will
build programs without PKCS #11.
## Without Docker
The requirements to build without Docker are:
1. Go version 1.5 is the minimum required version of Go. However, only Go 1.6+
is supported due to the test system not supporting Go 1.5.
2. A properly configured go environment
3. A properly configured GOPATH
4. With Go 1.5, you are required to set the environment variable
`GO15VENDOREXPERIMENT=1`.
Run:
$ go install github.com/cloudflare/cfssl/cmd/...
1.1.0 - 2015-08-04
ADDED:
- Revocation now checks OCSP status.
- Authenticated endpoints are now supported using HMAC tags.
- Bundle can verify certificates against a domain or IP.
- OCSP subcommand has been added.
- PKCS #11 keys are now supported; this support is now the default.
- OCSP serving is now implemented.
- The multirootca tool is now available for multiple signing
keys via an authenticated API.
- A scan utility for checking the quality of a server's TLS
configuration.
- The certificate bundler now supports PKCS #7 and PKCS #12.
- An info endpoint has been added to retrieve the signers'
certificates.
- Signers can now use a serial sequence number for certificate
serial numbers; the default remains randomised serial numbers.
- CSR whitelisting allows the signer to explicitly distrust
certain fields in a CSR.
- Signing profiles can include certificate policies and their
qualifiers.
- The multirootca can use Red October-secured private keys.
- The multirootca can whitelist CSRs per-signer based on an
IP network whitelist.
- The signer can whitelist SANs and common names via a regular-
expression whitelist.
- Multiple fallback remote signers are now supported in the
cfssl server.
- A Docker build script has been provided to facilitate building
CFSSL for all supported platforms.
- The log package includes a new logging level, fatal, that
immediately exits with error after printing the log message.
CHANGED:
- CLI tool can read from standard input.
- The -f flag has been renamed to -config.
- Signers have been refactored into local and remote signers
under a single universal signer abstraction.
- The CLI subcommands have been refactored into separate
packages.
- Signing can now extract subject information from a CSR.
- Various improvements to the certificate ubiquity scoring,
such as accounting for SHA1 deprecation.
- The bundle CLI tool can set the intermediates directory that
newly found intermediates can be stored in.
- The CLI tools return exit code 1 on failure.
CONTRIBUTORS:
Alice Xia
Dan Rohr
Didier Smith
Dominic Luechinger
Erik Kristensen
Fabian Ruff
George Tankersley
Harald Wagener
Harry Harpham
Jacob H. Haven
Jacob Hoffman-Andrews
Joshua Kroll
Kyle Isom
Nick Sullivan
Peter Eckersley
Richard Barnes
Sophie Huang
Steve Rude
Tara Vancil
Terin Stock
Thomaz Leite
Travis Truman
Zi Lin
FROM golang:1.9.2
ENV USER root
WORKDIR /go/src/github.com/cloudflare/cfssl
COPY . .
# restore all deps and build
RUN go get github.com/cloudflare/cfssl_trust/... && \
go get github.com/GeertJohan/go.rice/rice && \
rice embed-go -i=./cli/serve && \
cp -R /go/src/github.com/cloudflare/cfssl_trust /etc/cfssl && \
go install ./cmd/...
EXPOSE 8888
ENTRYPOINT ["cfssl"]
CMD ["--help"]
FROM golang:1.9.2
ENV USER root
WORKDIR /go/src/github.com/cloudflare/cfssl
COPY . .
# restore all deps and build
RUN go get github.com/mitchellh/gox
ENTRYPOINT ["gox"]
FROM golang:1.9.2-alpine3.6
ENV GOPATH /go
ENV USER root
COPY . /go/src/github.com/cloudflare/cfssl
RUN set -x && \
apk --no-cache add git gcc libc-dev && \
go get github.com/cloudflare/cfssl_trust/... && \
go get github.com/GeertJohan/go.rice/rice && \
cd /go/src/github.com/cloudflare/cfssl && rice embed-go -i=./cli/serve && \
mkdir bin && cd bin && \
go build ../cmd/cfssl && \
go build ../cmd/cfssljson && \
go build ../cmd/mkbundle && \
go build ../cmd/multirootca && \
echo "Build complete."
FROM alpine:3.6
COPY --from=0 /go/src/github.com/cloudflare/cfssl_trust /etc/cfssl
COPY --from=0 /go/src/github.com/cloudflare/cfssl/bin/ /usr/bin
VOLUME [ "/etc/cfssl" ]
WORKDIR /etc/cfssl
EXPOSE 8888
ENTRYPOINT ["cfssl"]
CMD ["--help"]
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "master"
name = "github.com/GeertJohan/go.rice"
packages = [
".",
"embedded"
]
revision = "c02ca9a983da5807ddf7d796784928f5be4afd09"
[[projects]]
name = "github.com/certifi/gocertifi"
packages = ["."]
revision = "deb3ae2ef2610fde3330947281941c562861188b"
version = "2018.01.18"
[[projects]]
branch = "master"
name = "github.com/cloudflare/backoff"
packages = ["."]
revision = "647f3cdfc87a18586e279c97afd6526d01b0d063"
[[projects]]
branch = "master"
name = "github.com/cloudflare/go-metrics"
packages = ["."]
revision = "6a9aea36fb410b71add0d0e58438e0cfc44d3076"
[[projects]]
branch = "master"
name = "github.com/cloudflare/redoctober"
packages = [
"client",
"config",
"core",
"cryptor",
"ecdh",
"hipchat",
"keycache",
"msp",
"order",
"padding",
"passvault",
"persist",
"report",
"symcrypt"
]
revision = "746a508df14c565d2074e2919d27c2efd1a7fe89"
[[projects]]
branch = "master"
name = "github.com/daaku/go.zipexe"
packages = ["."]
revision = "a5fe2436ffcb3236e175e5149162b41cd28bd27d"
[[projects]]
branch = "master"
name = "github.com/getsentry/raven-go"
packages = ["."]
revision = "563b81fc02b75d664e54da31f787c2cc2186780b"
[[projects]]
name = "github.com/go-sql-driver/mysql"
packages = ["."]
revision = "a0583e0143b1624142adab07e0e97fe106d99561"
version = "v1.3"
[[projects]]
name = "github.com/gogo/protobuf"
packages = ["proto"]
revision = "1adfc126b41513cc696b209667c8656ea7aac67c"
version = "v1.0.0"
[[projects]]
name = "github.com/golang/protobuf"
packages = [
"proto",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp"
]
revision = "925541529c1fa6821df4e44ce2723319eb2be768"
version = "v1.0.0"
[[projects]]
name = "github.com/google/certificate-transparency-go"
packages = [
".",
"asn1",
"client",
"client/configpb",
"jsonclient",
"tls",
"x509",
"x509/pkix"
]
revision = "5ab67e519c93568ac3ee50fd6772a5bcf8aa460d"
[[projects]]
name = "github.com/jmhodges/clock"
packages = ["."]
revision = "880ee4c335489bc78d01e4d0a254ae880734bc15"
version = "v1.1"
[[projects]]
branch = "master"
name = "github.com/jmoiron/sqlx"
packages = [
".",
"reflectx"
]
revision = "05cef0741ade10ca668982355b3f3f0bcf0ff0a8"
[[projects]]
branch = "master"
name = "github.com/kardianos/osext"
packages = ["."]
revision = "ae77be60afb1dcacde03767a8c37337fad28ac14"
[[projects]]
branch = "master"
name = "github.com/kisielk/sqlstruct"
packages = ["."]
revision = "648daed35d49dac24a4bff253b190a80da3ab6a5"
[[projects]]
name = "github.com/kisom/goutils"
packages = ["assert"]
revision = "d6c5360a068641fc799295411f429be80115bcfc"
version = "v1.1.0"
[[projects]]
branch = "master"
name = "github.com/lib/pq"
packages = [
".",
"oid"
]
revision = "88edab0803230a3898347e77b474f8c1820a1f20"
[[projects]]
name = "github.com/mattn/go-sqlite3"
packages = ["."]
revision = "6c771bb9887719704b210e87e934f08be014bdb1"
version = "v1.6.0"
[[projects]]
name = "github.com/pkg/errors"
packages = ["."]
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
branch = "master"
name = "golang.org/x/crypto"
packages = [
"cryptobyte",
"cryptobyte/asn1",
"ed25519",
"ed25519/internal/edwards25519",
"ocsp",
"pbkdf2",
"pkcs12",
"pkcs12/internal/rc2",
"scrypt"
]
revision = "c126467f60eb25f8f27e5a981f32a87e3965053f"
[[projects]]
branch = "master"
name = "golang.org/x/net"
packages = [
"context",
"context/ctxhttp"
]
revision = "f5dfe339be1d06f81b22525fe34671ee7d2c8904"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "eb3713a13a2a3e0a35c010aed983787299798de5f20ba838399802f9ae67107c"
solver-name = "gps-cdcl"
solver-version = 1
# Gopkg.toml example
#
# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
#
# [prune]
# non-go = false
# go-tests = true
# unused-packages = true
[[constraint]]
branch = "master"
name = "github.com/GeertJohan/go.rice"
[[constraint]]
branch = "master"
name = "github.com/cloudflare/backoff"
[[constraint]]
branch = "master"
name = "github.com/cloudflare/go-metrics"
[[constraint]]
branch = "master"
name = "github.com/cloudflare/redoctober"
[[constraint]]
name = "github.com/go-sql-driver/mysql"
version = "1.3.0"
[[constraint]]
name = "github.com/google/certificate-transparency-go"
revision = "5ab67e519c93568ac3ee50fd6772a5bcf8aa460d"
[[constraint]]
name = "github.com/jmhodges/clock"
version = "1.1.0"
[[constraint]]
branch = "master"
name = "github.com/jmoiron/sqlx"
[[constraint]]
branch = "master"
name = "github.com/kisielk/sqlstruct"
[[constraint]]
name = "github.com/kisom/goutils"
version = "1.1.0"
[[constraint]]
branch = "master"
name = "github.com/lib/pq"
[[constraint]]
name = "github.com/mattn/go-sqlite3"
version = "1.6.0"
[[constraint]]
branch = "master"
name = "golang.org/x/crypto"
[[constraint]]
branch = "master"
name = "golang.org/x/net"
[prune]
go-tests = true
unused-packages = true
#!/bin/bash
set -o errexit
CDIR=$(cd `dirname "$0"` && pwd)
cd "$CDIR"
ORG_PATH="github.com/cloudflare"
REPO_PATH="${ORG_PATH}/cfssl"
ARCH="$(uname -m)"
export GOPATH="${CDIR}/gopath"
export PATH="${PATH}:${GOPATH}/bin"
eval $(go env)
if [ ! -h gopath/src/${REPO_PATH} ]; then
mkdir -p gopath/src/${ORG_PATH}
ln -s ../../../.. gopath/src/${REPO_PATH} || exit 255
fi
ls "${GOPATH}/src/${REPO_PATH}"
PACKAGES=""
if [ "$#" != 0 ]; then
for pkg in "$@"; do
PACKAGES="$PACKAGES $REPO_PATH/$pkg"
done
else
PACKAGES=$(go list ./... | grep -v /vendor/ | grep ^_)
# Escape current cirectory
CDIR_ESC=$(printf "%q" "$CDIR/")
# Remove current directory from the package path
PACKAGES=${PACKAGES//$CDIR_ESC/}
# Remove underscores
PACKAGES=${PACKAGES//_/}
# split PACKAGES into an array and prepend REPO_PATH to each local package
split=(${PACKAGES// / })
PACKAGES=${split[@]/#/${REPO_PATH}/}
fi
# Build and install cfssl executable in PATH
go install -tags "$BUILD_TAGS" ${REPO_PATH}/cmd/cfssl
COVPROFILES=""
for package in $(go list -f '{{if len .TestGoFiles}}{{.ImportPath}}{{end}}' $PACKAGES)
do
profile="$GOPATH/src/$package/.coverprofile"
if [ $ARCH = 'x86_64' ]; then
go test -race -tags "$BUILD_TAGS" --coverprofile=$profile $package
else
go test -tags "$BUILD_TAGS" --coverprofile=$profile $package
fi
[ -s $profile ] && COVPROFILES="$COVPROFILES $profile"
done
cat $COVPROFILES > coverprofile.txt
if ! command -v fgt > /dev/null ; then
go get github.com/GeertJohan/fgt
fi
if ! command -v golint > /dev/null ; then
go get github.com/golang/lint/golint
fi
for package in $PACKAGES
do
echo "fgt golint ${package}"
fgt golint "${package}"
done
# Setting "sudo" to false forces Travis-CI to use its
# container-based build infrastructure, which has shorter
# queue times.
sudo: false
# Use the newer Travis-CI build templates based on the
# Debian Linux distribution "Trusty" release.
dist: trusty
# Selecting C as the language keeps the container to a
# minimum footprint.
language: c
jobs:
include:
# The test stage validates that the protobuf file was updated
# correctly prior to being committed.
- stage: test
script: make
# The lang stages validate the specification using
# language-specific bindings.
# Lang stage: Cxx
- stage: lang
script: make -C lib/cxx
# Lang stage: Go
- stage: lang
language: go
go: 1.10.4
go_import_path: github.com/container-storage-interface/spec
install:
- make -C lib/go protoc
- make -C lib/go protoc-gen-go
script:
- make -C lib/go
# How to Contribute
This document outlines some of the requirements and conventions for contributing to the Container Storage Interface, including development workflow, commit message formatting, contact points, and other resources to make it easier to get your contribution accepted.
## Licence and CLA
CSI is under [Apache 2.0](LICENSE) and accepts contributions via GitHub pull requests.
Before contributing to the Container Storage Interface, contributors MUST sign the CLA available [here](https://github.com/container-storage-interface/spec/blob/master/CCLA.pdf).
The CLA MAY be signed on behalf of a company, possibly covering multiple contributors, or as an individual (put "Individual" for "Corporation name").
The completed CLA MUST be mailed to the CSI Approvers [mailing list](container-storage-interface-approvers@googlegroups.com).
## Markdown style
To keep consistency throughout the Markdown files in the CSI spec, all files should be formatted one sentence per line.
This fixes two things: it makes diffing easier with git and it resolves fights about line wrapping length.
For example, this paragraph will span three lines in the Markdown source.
## Code style
This also applies to the code snippets in the markdown files.
* Please wrap the code at 72 characters.
## Comments
This also applies to the code snippets in the markdown files.
* End each sentence within a comment with a punctuation mark (please note that we generally prefer periods); this applies to incomplete sentences as well.
* For trailing comments, leave one space between the end of the code and the beginning of the comment.
## Git commit
The "system of record" for the specification is the `spec.md` file and all hand-edits of the specification should happen there.
**DO NOT** manually edit the generated protobufs or generated language bindings.
Once changes to `spec.md` are complete, please run `make` to update generated files.
**IMPORTANT:** Prior to committing code please run `make` to ensure that your specification changes have landed in all generated files.
### Commit Style
Each commit should represent a single logical (atomic) change: this makes your changes easier to review.
* Try to avoid unrelated cleanups (e.g., typo fixes or style nits) in the same commit that makes functional changes.
While typo fixes are great, including them in the same commit as functional changes makes the commit history harder to read.
* Developers often make incremental commits to save their progress when working on a change, and then “rewrite history” (e.g., using `git rebase -i`) to create a clean set of commits once the change is ready to be reviewed.
Simple house-keeping for clean git history.
Read more on [How to Write a Git Commit Message](http://chris.beams.io/posts/git-commit/) or the Discussion section of [`git-commit(1)`](http://git-scm.com/docs/git-commit).
* Separate the subject from body with a blank line.
* Limit the subject line to 50 characters.
* Capitalize the subject line.
* Do not end the subject line with a period.
* Use the imperative mood in the subject line.
* Wrap the body at 72 characters.
* Use the body to explain what and why vs. how.
* If there was important/useful/essential conversation or information, copy or include a reference.
* When possible, one keyword to scope the change in the subject (i.e. "README: ...", "tool: ...").
all: build
CSI_SPEC := spec.md
CSI_PROTO := csi.proto
# This is the target for building the temporary CSI protobuf file.
#
# The temporary file is not versioned, and thus will always be
# built on Travis-CI.
$(CSI_PROTO).tmp: $(CSI_SPEC) Makefile
echo "// Code generated by make; DO NOT EDIT." > "$@"
cat $< | sed -n -e '/```protobuf$$/,/^```$$/ p' | sed '/^```/d' >> "$@"
# This is the target for building the CSI protobuf file.
#
# This target depends on its temp file, which is not versioned.
# Therefore when built on Travis-CI the temp file will always
# be built and trigger this target. On Travis-CI the temp file
# is compared with the real file, and if they differ the build
# will fail.
#
# Locally the temp file is simply copied over the real file.
$(CSI_PROTO): $(CSI_PROTO).tmp
ifeq (true,$(TRAVIS))
diff "$@" "$?"
else
diff "$@" "$?" > /dev/null 2>&1 || cp -f "$?" "$@"
endif
build: check
# If this is not running on Travis-CI then for sake of convenience
# go ahead and update the language bindings as well.
ifneq (true,$(TRAVIS))
build:
$(MAKE) -C lib/go
$(MAKE) -C lib/cxx
endif
clean:
$(MAKE) -C lib/go $@
clobber: clean
$(MAKE) -C lib/go $@
rm -f $(CSI_PROTO) $(CSI_PROTO).tmp
# check generated files for violation of standards
check: $(CSI_PROTO)
awk '{ if (length > 72) print NR, $$0 }' $? | diff - /dev/null
.PHONY: clean clobber check
approvers:
- saad-ali # Representing Kubernetes
- thockin # Representing Kubernetes
- jieyu # Representing Mesos
- jdef # Representing Mesos
- anusha-ragunathan # Representing Docker
- ddebroy # Representing Docker
- julian-hj # Representing Cloud Foundry
- paulcwarren # Representing Cloud Foundry
reviewers:
# Container Storage Interface (CSI) Specification [![build status](https://travis-ci.org/container-storage-interface/spec.svg?branch=master)](https://travis-ci.org/container-storage-interface/spec)
![CSI Logo](logo.png)
This project contains the CSI [specification](spec.md) and [protobuf](csi.proto) files.
## CSI Adoption
### Container Orchestrators (CO)
* [Cloud Foundry](https://github.com/cloudfoundry/csi-plugins-release/blob/master/CSI_SUPPORT.md)
* [Kubernetes](https://kubernetes-csi.github.io/docs/)
* [Mesos](http://mesos.apache.org/documentation/latest/csi/)
# CSI Validation Libraries
This directory contains language bindings generated from the CSI [protobuf file](../csi.proto) used to validate the model and workflows of the CSI specification.
all: build
########################################################################
## GOLANG ##
########################################################################
# If GOPATH isn't defined then set its default location.
ifeq (,$(strip $(GOPATH)))
GOPATH := $(HOME)/go
else
# If GOPATH is already set then update GOPATH to be its own
# first element.
GOPATH := $(word 1,$(subst :, ,$(GOPATH)))
endif
export GOPATH
########################################################################
## PROTOC ##
########################################################################
# Only set PROTOC_VER if it has an empty value.
ifeq (,$(strip $(PROTOC_VER)))
PROTOC_VER := 3.5.1
endif
PROTOC_OS := $(shell uname -s)
ifeq (Darwin,$(PROTOC_OS))
PROTOC_OS := osx
endif
PROTOC_ARCH := $(shell uname -m)
ifeq (i386,$(PROTOC_ARCH))
PROTOC_ARCH := x86_32
endif
PROTOC := ./protoc
PROTOC_ZIP := protoc-$(PROTOC_VER)-$(PROTOC_OS)-$(PROTOC_ARCH).zip
PROTOC_URL := https://github.com/google/protobuf/releases/download/v$(PROTOC_VER)/$(PROTOC_ZIP)
PROTOC_TMP_DIR := .protoc
PROTOC_TMP_BIN := $(PROTOC_TMP_DIR)/bin/protoc
$(PROTOC):
-mkdir -p "$(PROTOC_TMP_DIR)" && \
curl -L $(PROTOC_URL) -o "$(PROTOC_TMP_DIR)/$(PROTOC_ZIP)" && \
unzip "$(PROTOC_TMP_DIR)/$(PROTOC_ZIP)" -d "$(PROTOC_TMP_DIR)" && \
chmod 0755 "$(PROTOC_TMP_BIN)" && \
cp -f "$(PROTOC_TMP_BIN)" "$@"
stat "$@" > /dev/null 2>&1
########################################################################
## PROTOC-GEN-GO ##
########################################################################
# This is the recipe for getting and installing the go plug-in
# for protoc
PROTOC_GEN_GO_PKG := github.com/golang/protobuf/protoc-gen-go
PROTOC_GEN_GO := protoc-gen-go
$(PROTOC_GEN_GO): PROTOBUF_PKG := $(dir $(PROTOC_GEN_GO_PKG))
$(PROTOC_GEN_GO): PROTOBUF_VERSION := v1.2.0
$(PROTOC_GEN_GO):
mkdir -p $(dir $(GOPATH)/src/$(PROTOBUF_PKG))
test -d $(GOPATH)/src/$(PROTOBUF_PKG)/.git || git clone https://$(PROTOBUF_PKG) $(GOPATH)/src/$(PROTOBUF_PKG)
(cd $(GOPATH)/src/$(PROTOBUF_PKG) && \
(test "$$(git describe --tags | head -1)" = "$(PROTOBUF_VERSION)" || \
(git fetch && git checkout tags/$(PROTOBUF_VERSION))))
(cd $(GOPATH)/src/$(PROTOBUF_PKG) && go get -v -d $$(go list -f '{{ .ImportPath }}' ./...)) && \
go build -o "$@" $(PROTOC_GEN_GO_PKG)
########################################################################
## PATH ##
########################################################################
# Update PATH with the current directory. This enables the protoc
# binary to discover the protoc-gen-go binary, built inside this
# directory.
export PATH := $(shell pwd):$(PATH)
########################################################################
## BUILD ##
########################################################################
CSI_PROTO := ../../csi.proto
CSI_PKG_ROOT := github.com/container-storage-interface/spec
CSI_PKG_SUB := $(shell cat $(CSI_PROTO) | sed -n -e 's/^package.\([^;]*\).v[0-9]\+;$$/\1/p'|tr '.' '/')
CSI_BUILD := $(CSI_PKG_SUB)/.build
CSI_GO := $(CSI_PKG_SUB)/csi.pb.go
CSI_A := csi.a
CSI_GO_TMP := $(CSI_BUILD)/$(CSI_PKG_ROOT)/csi.pb.go
# This recipe generates the go language bindings to a temp area.
$(CSI_GO_TMP): HERE := $(shell pwd)
$(CSI_GO_TMP): PTYPES_PKG := github.com/golang/protobuf/ptypes
$(CSI_GO_TMP): GO_OUT := plugins=grpc
$(CSI_GO_TMP): GO_OUT := $(GO_OUT),Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor
$(CSI_GO_TMP): GO_OUT := $(GO_OUT),Mgoogle/protobuf/wrappers.proto=$(PTYPES_PKG)/wrappers
$(CSI_GO_TMP): GO_OUT := $(GO_OUT):"$(HERE)/$(CSI_BUILD)"
$(CSI_GO_TMP): INCLUDE := -I$(GOPATH)/src -I$(HERE)/$(PROTOC_TMP_DIR)/include
$(CSI_GO_TMP): $(CSI_PROTO) | $(PROTOC) $(PROTOC_GEN_GO)
@mkdir -p "$(@D)"
(cd "$(GOPATH)/src" && \
$(HERE)/$(PROTOC) $(INCLUDE) --go_out=$(GO_OUT) "$(CSI_PKG_ROOT)/$(<F)")
# The temp language bindings are compared to the ones that are
# versioned. If they are different then it means the language
# bindings were not updated prior to being committed.
$(CSI_GO): $(CSI_GO_TMP)
ifeq (true,$(TRAVIS))
diff "$@" "$?"
else
@mkdir -p "$(@D)"
diff "$@" "$?" > /dev/null 2>&1 || cp -f "$?" "$@"
endif
# This recipe builds the Go archive from the sources in three steps:
#
# 1. Go get any missing dependencies.
# 2. Cache the packages.
# 3. Build the archive file.
$(CSI_A): $(CSI_GO)
go get -v -d ./...
go install ./$(CSI_PKG_SUB)
go build -o "$@" ./$(CSI_PKG_SUB)
build: $(CSI_A)
clean:
go clean -i ./...
rm -rf "$(CSI_A)" "$(CSI_GO)" "$(CSI_BUILD)"
clobber: clean
rm -fr "$(PROTOC)" "$(PROTOC_GEN_GO)" "$(CSI_PKG_SUB)"
.PHONY: clean clobber
# CSI Go Validation
This package is used to validate the CSI specification with Go language bindings.
## Build Reference
To validate the Go language bindings against the current specification use the following command:
```bash
$ make
```
The above command will download the `protoc` and `protoc-gen-go` binaries if they are not present and then proceed to build the CSI Go language bindings.
### Environment Variables
The following table lists the environment variables that can be used to influence the behavior of the Makefile:
| Name | Default Value | Description |
|------|---------------|-------------|
| `PROTOC_VER` | `3.3.0` | The version of the protoc binary. |
This source diff could not be displayed because it is too large. You can view the blob instead.
File mode changed from 100755 to 100644
This directory contains the GRPC API definitions for containerd.
All defined services and messages have been aggregated into `*.pb.txt`
descriptors files in this directory. Definitions present here are considered
frozen after the release.
At release time, the current `next.pb.txt` file will be moved into place to
freeze the API changes for the minor version. For example, when 1.0.0 is
released, `next.pb.txt` should be moved to `1.0.txt`. Notice that we leave off
the patch number, since the API will be completely locked down for a given
patch series.
We may find that by default, protobuf descriptors are too noisy to lock down
API changes. In that case, we may filter out certain fields in the descriptors,
possibly regenerating for old versions.
This process is similar to the [process used to ensure backwards compatibility
in Go](https://github.com/golang/go/tree/master/api).
# This dockerfile is used to test containerd within a container
#
# usage:
# 1.) docker build -t containerd-test -f Dockerfile.test ../
# 2.) docker run -it --privileged -v /tmp:/tmp --tmpfs /var/lib/containerd-test containerd-test bash
# 3.) $ make binaries install test
#
ARG GOLANG_VERSION=1.12
FROM golang:${GOLANG_VERSION} AS golang-base
RUN mkdir -p /go/src/github.com/containerd/containerd
WORKDIR /go/src/github.com/containerd/containerd
# Install proto3
FROM golang-base AS proto3
RUN apt-get update && apt-get install -y \
autoconf \
automake \
g++ \
libtool \
unzip \
--no-install-recommends
COPY script/setup/install-protobuf install-protobuf
RUN ./install-protobuf
# Install runc
FROM golang-base AS runc
RUN apt-get update && apt-get install -y \
curl \
libseccomp-dev \
--no-install-recommends
COPY vendor.conf vendor.conf
COPY script/setup/install-runc install-runc
RUN ./install-runc
FROM golang-base AS dev
RUN apt-get update && apt-get install -y \
btrfs-tools \
gcc \
git \
libseccomp-dev \
make \
xfsprogs \
--no-install-recommends
COPY --from=proto3 /usr/local/bin/protoc /usr/local/bin/protoc
COPY --from=proto3 /usr/local/include/google /usr/local/include/google
COPY --from=runc /usr/local/sbin/runc /usr/local/go/bin/runc
COPY . .
# contrib
The `contrib` directory contains packages that do not belong in the core containerd packages but still contribute to overall containerd usability.
Package such as Apparmor or Selinux are placed in `contrib` because they are platform dependent and often require higher level tools and profiles to work.
Packaging and other built tools can be added to `contrib` to aid in packaging containerd for various distributions.
## Testing
Code in the `contrib` directory may or may not have been tested in the normal test pipeline for core components.
File mode changed from 100755 to 100644
......@@ -4,7 +4,17 @@ go:
- 1.10.x
- tip
install:
- go get -t -v ./...
- go get -u github.com/vbatts/git-validation
- go get -u github.com/kunalkushwaha/ltag
before_script:
- pushd ..; git clone https://github.com/containerd/project; popd
script:
- DCO_VERBOSITY=-q ../project/script/validate/dco
- ../project/script/validate/fileheader ../project/
- go test -race -coverprofile=coverage.txt -covermode=atomic
after_success:
......
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
https://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
......@@ -175,24 +176,13 @@
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Copyright The containerd Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
......
......@@ -7,3 +7,13 @@
A Go package for managing the registration, marshaling, and unmarshaling of encoded types.
This package helps when types are sent over a GRPC API and marshaled as a [protobuf.Any]().
## Project details
**typeurl** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE).
As a containerd sub-project, you will find the:
* [Project governance](https://github.com/containerd/project/blob/master/GOVERNANCE.md),
* [Maintainers](https://github.com/containerd/project/blob/master/MAINTAINERS),
* and [Contributing guidelines](https://github.com/containerd/project/blob/master/CONTRIBUTING.md)
information in our [`containerd/project`](https://github.com/containerd/project) repository.
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package typeurl
// Package typeurl assists with managing the registration, marshaling, and
// unmarshaling of types encoded as protobuf.Any.
//
// A protobuf.Any is a proto message that can contain any arbitrary data. It
// consists of two components, a TypeUrl and a Value, and its proto definition
// looks like this:
//
// message Any {
// string type_url = 1;
// bytes value = 2;
// }
//
// The TypeUrl is used to distinguish the contents from other proto.Any
// messages. This typeurl library manages these URLs to enable automagic
// marshaling and unmarshaling of the contents.
//
// For example, consider this go struct:
//
// type Foo struct {
// Field1 string
// Field2 string
// }
//
// To use typeurl, types must first be registered. This is typically done in
// the init function
//
// func init() {
// typeurl.Register(&Foo{}, "Foo")
// }
//
// This will register the type Foo with the url path "Foo". The arguments to
// Register are variadic, and are used to construct a url path. Consider this
// example, from the github.com/containerd/containerd/client package:
//
// func init() {
// const prefix = "types.containerd.io"
// // register TypeUrls for commonly marshaled external types
// major := strconv.Itoa(specs.VersionMajor)
// typeurl.Register(&specs.Spec{}, prefix, "opencontainers/runtime-spec", major, "Spec")
// // this function has more Register calls, which are elided.
// }
//
// This registers several types under a more complex url, which ends up mapping
// to `types.containerd.io/opencontainers/runtime-spec/1/Spec` (or some other
// value for major).
//
// Once a type is registered, it can be marshaled to a proto.Any message simply
// by calling `MarshalAny`, like this:
//
// foo := &Foo{Field1: "value1", Field2: "value2"}
// anyFoo, err := typeurl.MarshalAny(foo)
//
// MarshalAny will resolve the correct URL for the type. If the type in
// question implements the proto.Message interface, then it will be marshaled
// as a proto message. Otherwise, it will be marshaled as json. This means that
// typeurl will work on any arbitrary data, whether or not it has a proto
// definition, as long as it can be serialized to json.
//
// To unmarshal, the process is simply inverse:
//
// iface, err := typeurl.UnmarshalAny(anyFoo)
// foo := iface.(*Foo)
//
// The correct type is automatically chosen from the type registry, and the
// returned interface can be cast straight to that type.
......@@ -78,7 +78,10 @@ func Is(any *types.Any, v interface{}) bool {
return any.TypeUrl == url
}
// MarshalAny marshals the value v into an any with the correct TypeUrl
// MarshalAny marshals the value v into an any with the correct TypeUrl.
// If the provided object is already a proto.Any message, then it will be
// returned verbatim. If it is of type proto.Message, it will be marshaled as a
// protocol buffer. Otherwise, the object will be marshaled to json.
func MarshalAny(v interface{}) (*types.Any, error) {
var marshal func(v interface{}) ([]byte, error)
switch t := v.(type) {
......
bin/
gopath/
*.sw[ponm]
.vagrant
release-*
language: go
sudo: required
dist: trusty
go:
- 1.7.x
- 1.8.x
env:
global:
- TOOLS_CMD=golang.org/x/tools/cmd
- PATH=$GOROOT/bin:$PATH
- GO15VENDOREXPERIMENT=1
matrix:
- TARGET=amd64
- TARGET=arm
- TARGET=arm64
- TARGET=ppc64le
- TARGET=s390x
matrix:
fast_finish: true
install:
- go get ${TOOLS_CMD}/cover
- go get github.com/modocache/gover
- go get github.com/mattn/goveralls
script:
- >
if [ "${TARGET}" == "amd64" ]; then
GOARCH="${TARGET}" ./test.sh;
else
GOARCH="${TARGET}" ./build.sh;
fi
notifications:
email: false
git:
depth: 9999999
## Community Code of Conduct
CNI follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
# How to Contribute
CNI is [Apache 2.0 licensed](LICENSE) and accepts contributions via GitHub
pull requests. This document outlines some of the conventions on development
workflow, commit message formatting, contact points and other resources to make
it easier to get your contribution accepted.
We gratefully welcome improvements to documentation as well as to code.
# Certificate of Origin
By contributing to this project you agree to the Developer Certificate of
Origin (DCO). This document was created by the Linux Kernel community and is a
simple statement that you, as a contributor, have the legal right to make the
contribution. See the [DCO](DCO) file for details.
# Email and Chat
The project uses the the cni-dev email list and IRC chat:
- Email: [cni-dev](https://groups.google.com/forum/#!forum/cni-dev)
- IRC: #[containernetworking](irc://irc.freenode.org:6667/#containernetworking) channel on freenode.org
Please avoid emailing maintainers found in the MAINTAINERS file directly. They
are very busy and read the mailing lists.
## Getting Started
- Fork the repository on GitHub
- Read the [README](README.md) for build and test instructions
- Play with the project, submit bugs, submit pull requests!
## Contribution workflow
This is a rough outline of how to prepare a contribution:
- Create a topic branch from where you want to base your work (usually branched from master).
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- If you changed code:
- add automated tests to cover your changes, using the [Ginkgo](http://onsi.github.io/ginkgo/) & [Gomega](http://onsi.github.io/gomega/) style
- if the package did not previously have any test coverage, add it to the list
of `TESTABLE` packages in the `test.sh` script.
- run the full test script and ensure it passes
- Make sure any new code files have a license header (this is now enforced by automated tests)
- Submit a pull request to the original repository.
## How to run the test suite
We generally require test coverage of any new features or bug fixes.
Here's how you can run the test suite on any system (even Mac or Windows) using
[Vagrant](https://www.vagrantup.com/) and a hypervisor of your choice:
```bash
vagrant up
vagrant ssh
# you're now in a shell in a virtual machine
sudo su
cd /go/src/github.com/containernetworking/cni
# to run the full test suite
./test.sh
# to focus on a particular test suite
cd plugins/main/loopback
go test
```
# Acceptance policy
These things will make a PR more likely to be accepted:
* a well-described requirement
* tests for new code
* tests for old code!
* new code and tests follow the conventions in old code and tests
* a good commit message (see below)
In general, we will merge a PR once two maintainers have endorsed it.
Trivial changes (e.g., corrections to spelling) may get waved through.
For substantial changes, more people may become involved, and you might get asked to resubmit the PR or divide the changes into more than one PR.
### Format of the Commit Message
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Fixes #38
```
The format can be described more formally as follows:
```
<subsystem>: <what changed>
<BLANK LINE>
<why this change was made>
<BLANK LINE>
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
## 3rd party plugins
So you've built a CNI plugin. Where should it live?
Short answer: We'd be happy to link to it from our [list of 3rd party plugins](README.md#3rd-party-plugins).
But we'd rather you kept the code in your own repo.
Long answer: An advantage of the CNI model is that independent plugins can be
built, distributed and used without any code changes to this repository. While
some widely used plugins (and a few less-popular legacy ones) live in this repo,
we're reluctant to add more.
If you have a good reason why the CNI maintainers should take custody of your
plugin, please open an issue or PR.
# Extension conventions
There are three ways of passing information to plugins using the Container Network Interface (CNI), none of which require the [spec](SPEC.md) to be updated. These are
- plugin specific fields in the JSON config
- `args` field in the JSON config
- `CNI_ARGS` environment variable
This document aims to provide guidance on which method should be used and to provide a convention for how common information should be passed.
Establishing these conventions allows plugins to work across multiple runtimes. This helps both plugins and the runtimes.
## Plugins
* Plugin authors should aim to support these conventions where it makes sense for their plugin. This means they are more likely to "just work" with a wider range of runtimes.
* Plugins should accept arguments according to these conventions if they implement the same basic functionality as other plugins. If plugins have shared functionality that isn't coverered by these conventions then a PR should be opened against this document.
## Runtimes
* Runtime authors should follow these conventions if they want to pass additional information to plugins. This will allow the extra information to be consumed by the widest range of plugins.
* These conventions serve as an abstraction for the runtime. For example, port forwarding is highly implementation specific, but users should be able to select the plugin of their choice without changing the runtime.
# Current conventions
Additional conventions can be created by creating PRs which modify this document.
## Plugin specific fields
[Plugin specific fields](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration) formed part of the original CNI spec and have been present since the initial release.
> Plugins may define additional fields that they accept and may generate an error if called with unknown fields. The exception to this is the args field may be used to pass arbitrary data which may be ignored by plugins.
A plugin can define any additional fields it needs to work properly. It is expected that it will return an error if it can't act on fields that were expected or where the field values were malformed.
This method of passing information to a plugin is recommended when the following conditions hold
* The configuration has specific meaning to the plugin (i.e. it's not just general meta data)
* the plugin is expected to act on the configuration or return an error if it can't
Dynamic information (i.e. data that a runtime fills out) should be placed in a `runtimeConfig` section.
| Area | Purpose | Spec and Example | Runtime implementations | Plugin Implementations |
| ----- | ------- | ---------------- | ----------------------- | --------------------- |
| port mappings | Pass mapping from ports on the host to ports in the container network namespace. | Operators can ask runtimes to pass port mapping information to plugins, by setting the following in the CNI config <pre>"capabilities": {"portMappings": true} </pre> Runtimes should fill in the actual port mappings when the config is passed to plugins. It should be placed in a new section of the config "runtimeConfig" e.g. <pre>"runtimeConfig": {<br /> "portMappings" : [<br /> { "hostPort": 8080, "containerPort": 80, "protocol": "tcp" },<br /> { "hostPort": 8000, "containerPort": 8001, "protocol": "udp" }<br /> ]<br />}</pre> | none | none |
For example, the configuration for a port mapping plugin might look like this to an operator (it should be included as part of a [network configuration list](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-lists).
```json
{
"name" : "ExamplePlugin",
"type" : "port-mapper",
"capabilities": {"portMappings": true}
}
```
But the runtime would fill in the mappings so the plugin itself would receive something like this.
```json
{
"name" : "ExamplePlugin",
"type" : "port-mapper",
"runtimeConfig": {
"portMappings": [
{"hostPort": 8080, "containerPort": 80, "protocol": "tcp"}
]
}
}
```
## "args" in network config
`args` in [network config](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration) were introduced as an optional field into the `0.2.0` release of the CNI spec. The first CNI code release that it appeared in was `v0.4.0`.
> args (dictionary): Optional additional arguments provided by the container runtime. For example a dictionary of labels could be passed to CNI plugins by adding them to a labels field under args.
`args` provide a way of providing more structured data than the flat strings that CNI_ARGS can support.
`args` should be used for _optional_ meta-data. Runtimes can place additional data in `args` and plugins that don't understand that data should just ignore it. Runtimes should not require that a plugin understands or consumes that data provided, and so a runtime should not expect to receive an error if the data could not be acted on.
This method of passing information to a plugin is recommended when the information is optional and the plugin can choose to ignore it. It's often that case that such information is passed to all plugins by the runtime whithout regard for whether the plugin can understand it.
The conventions documented here are all namepaced under `cni` so they don't conflict with any existing `args`.
For example:
```json
{
"cniVersion":"0.2.0",
"name":"net",
"args":{
"cni":{
"labels": [{"key": "app", "value": "myapp"}]
}
},
<REST OF CNI CONFIG HERE>
"ipam":{
<IPAM CONFIG HERE>
}
}
```
| Area | Purpose| Spec and Example | Runtime implementations | Plugin Implementations |
| ----- | ------ | ------------ | ----------------------- | ---------------------- |
| labels | Pass`key=value` labels to plugins | <pre>"labels" : [<br /> { "key" : "app", "value" : "myapp" },<br /> { "key" : "env", "value" : "prod" }<br />] </pre> | none | none |
## CNI_ARGS
CNI_ARGS formed part of the original CNI spec and have been present since the initial release.
> `CNI_ARGS`: Extra arguments passed in by the user at invocation time. Alphanumeric key-value pairs separated by semicolons; for example, "FOO=BAR;ABC=123"
The use of `CNI_ARGS` is deprecated and "args" should be used instead.
| Field | Purpose| Spec and Example | Runtime implementations | Plugin Implementations |
| ------ | ------ | ---------------- | ----------------------- | ---------------------- |
| IP | Request a specific IP from IPAM plugins | IP=192.168.10.4 | *rkt* supports passing additional arguments to plugins and the [documentation](https://coreos.com/rkt/docs/latest/networking/overriding-defaults.html) suggests IP can be used. | host-local (since version v0.2.0) supports the field for IPv4 only - [documentation](https://github.com/containernetworking/cni/blob/master/Documentation/host-local.md#supported-arguments).|
## Chained Plugins
If plugins are agnostic about the type of interface created, they SHOULD work in a chained mode and configure existing interfaces. Plugins MAY also create the desired interface when not run in a chain.
For example, the `bridge` plugin adds the host-side interface to a bridge. So, it should accept any previous result that includes a host-side interface, including `tap` devices. If not called as a chained plugin, it creates a `veth` pair first.
Plugins that meet this convention are usable by a larger set of runtimes and interfaces, including hypervisors and DPDK providers.
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
Bryan Boreham <bryan@weave.works> (@bboreham)
Casey Callendrello <casey.callendrello@coreos.com> (@squeed)
Dan Williams <dcbw@redhat.com> (@dcbw)
Gabe Rosenhouse <grosenhouse@pivotal.io> (@rosenhouse)
Stefan Junker <stefan.junker@coreos.com> (@steveeJ)
Tom Denham <tom@tigera.io> (@tomdee)
[![Build Status](https://travis-ci.org/containernetworking/cni.svg?branch=master)](https://travis-ci.org/containernetworking/cni)
[![Coverage Status](https://coveralls.io/repos/github/containernetworking/cni/badge.svg?branch=master)](https://coveralls.io/github/containernetworking/cni?branch=master)
[![Slack Status](https://cryptic-tundra-43194.herokuapp.com/badge.svg)](https://cryptic-tundra-43194.herokuapp.com/)
![CNI Logo](logo.png)
---
# Community Sync Meeting
There is a community sync meeting for users and developers every 1-2 months. The next meeting will help on a Google Hangout and the link is in the [agenda](https://docs.google.com/document/d/10ECyT2mBGewsJUcmYmS8QNo1AcNgy2ZIe2xS7lShYhE/edit?usp=sharing) (Notes from previous meeting are also in this doc). The next meeting will be held on *Wednesday, June 21th* at *3:00pm UTC* [Add to Calendar]https://www.worldtimebuddy.com/?qm=1&lid=100,5,2643743,5391959&h=100&date=2017-6-21&sln=15-16).
---
# CNI - the Container Network Interface
## What is CNI?
CNI (_Container Network Interface_), a [Cloud Native Computing Foundation](https://cncf.io) project, consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of supported plugins.
CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.
Because of this focus, CNI has a wide range of support and the specification is simple to implement.
As well as the [specification](SPEC.md), this repository contains the Go source code of a [library for integrating CNI into applications](libcni) and an [example command-line tool](cnitool) for executing CNI plugins. A [separate repository contains reference plugins](https://github.com/containernetworking/plugins) and a template for making new plugins.
The template code makes it straight-forward to create a CNI plugin for an existing container networking project.
CNI also makes a good framework for creating a new container networking project from scratch.
## Why develop CNI?
Application containers on Linux are a rapidly evolving area, and within this area networking is not well addressed as it is highly environment-specific.
We believe that many container runtimes and orchestrators will seek to solve the same problem of making the network layer pluggable.
To avoid duplication, we think it is prudent to define a common interface between the network plugins and container execution: hence we put forward this specification, along with libraries for Go and a set of plugins.
## Who is using CNI?
### Container runtimes
- [rkt - container engine](https://coreos.com/blog/rkt-cni-networking.html)
- [Kurma - container runtime](http://kurma.io/)
- [Kubernetes - a system to simplify container operations](http://kubernetes.io/docs/admin/network-plugins/)
- [OpenShift - Kubernetes with additional enterprise features](https://github.com/openshift/origin/blob/master/docs/openshift_networking_requirements.md)
- [Cloud Foundry - a platform for cloud applications](https://github.com/cloudfoundry-incubator/cf-networking-release)
- [Mesos - a distributed systems kernel](https://github.com/apache/mesos/blob/master/docs/cni.md)
### 3rd party plugins
- [Project Calico - a layer 3 virtual network](https://github.com/projectcalico/calico-cni)
- [Weave - a multi-host Docker network](https://github.com/weaveworks/weave)
- [Contiv Networking - policy networking for various use cases](https://github.com/contiv/netplugin)
- [SR-IOV](https://github.com/hustcat/sriov-cni)
- [Cilium - BPF & XDP for containers](https://github.com/cilium/cilium)
- [Infoblox - enterprise IP address management for containers](https://github.com/infobloxopen/cni-infoblox)
- [Multus - a Multi plugin](https://github.com/Intel-Corp/multus-cni)
- [Romana - Layer 3 CNI plugin supporting network policy for Kubernetes](https://github.com/romana/kube)
- [CNI-Genie - generic CNI network plugin](https://github.com/Huawei-PaaS/CNI-Genie)
- [Nuage CNI - Nuage Networks SDN plugin for network policy kubernetes support ](https://github.com/nuagenetworks/nuage-cni)
- [Silk - a CNI plugin designed for Cloud Foundry](https://github.com/cloudfoundry-incubator/silk)
- [Linen - a CNI plugin designed for overlay networks with Open vSwitch and fit in SDN/OpenFlow network environment](https://github.com/John-Lin/linen-cni)
The CNI team also maintains some [core plugins in a separate repository](https://github.com/containernetworking/plugins).
## Contributing to CNI
We welcome contributions, including [bug reports](https://github.com/containernetworking/cni/issues), and code and documentation improvements.
If you intend to contribute to code or documentation, please read [CONTRIBUTING.md](CONTRIBUTING.md). Also see the [contact section](#contact) in this README.
## How do I use CNI?
### Requirements
The CNI spec is language agnostic. To use the Go language libraries in this repository, you'll need a recent version of Go. Our [automated tests](https://travis-ci.org/containernetworking/cni/builds) cover Go versions 1.7 and 1.8.
### Reference Plugins
The CNI project maintains a set of [reference plugins](https://github.com/containernetworking/plugins) that implement the CNI specification.
NOTE: the reference plugins used to live in this repository but have been split out into a [separate repository](https://github.com/containernetworking/plugins) as of May 2017.
### Running the plugins
After building and installing the [reference plugins](https://github.com/containernetworking/plugins), you can use the `priv-net-run.sh` and `docker-run.sh` scripts in the `scripts/` directory to exercise the plugins.
**note - priv-net-run.sh depends on `jq`**
Start out by creating a netconf file to describe a network:
```bash
$ mkdir -p /etc/cni/net.d
$ cat >/etc/cni/net.d/10-mynet.conf <<EOF
{
"cniVersion": "0.2.0",
"name": "mynet",
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"subnet": "10.22.0.0/16",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
}
EOF
$ cat >/etc/cni/net.d/99-loopback.conf <<EOF
{
"cniVersion": "0.2.0",
"type": "loopback"
}
EOF
```
The directory `/etc/cni/net.d` is the default location in which the scripts will look for net configurations.
Next, build the plugins:
```bash
$ cd $GOPATH/src/github.com/containernetworking/plugins
$ ./build.sh
```
Finally, execute a command (`ifconfig` in this example) in a private network namespace that has joined the `mynet` network:
```bash
$ CNI_PATH=$GOPATH/src/github.com/containernetworking/plugins/bin
$ cd $GOPATH/src/github.com/containernetworking/cni/scripts
$ sudo CNI_PATH=$CNI_PATH ./priv-net-run.sh ifconfig
eth0 Link encap:Ethernet HWaddr f2:c2:6f:54:b8:2b
inet addr:10.22.0.2 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::f0c2:6fff:fe54:b82b/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:90 (90.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
```
The environment variable `CNI_PATH` tells the scripts and library where to look for plugin executables.
## Running a Docker container with network namespace set up by CNI plugins
Use the instructions in the previous section to define a netconf and build the plugins.
Next, docker-run.sh script wraps `docker run`, to execute the plugins prior to entering the container:
```bash
$ CNI_PATH=$GOPATH/src/github.com/containernetworking/plugins/bin
$ cd $GOPATH/src/github.com/containernetworking/cni/scripts
$ sudo CNI_PATH=$CNI_PATH ./docker-run.sh --rm busybox:latest ifconfig
eth0 Link encap:Ethernet HWaddr fa:60:70:aa:07:d1
inet addr:10.22.0.2 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::f860:70ff:feaa:7d1/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:90 (90.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
```
## What might CNI do in the future?
CNI currently covers a wide range of needs for network configuration due to it simple model and API.
However, in the future CNI might want to branch out into other directions:
- Dynamic updates to existing network configuration
- Dynamic policies for network bandwidth and firewall rules
If these topics of are interest, please contact the team via the mailing list or IRC and find some like-minded people in the community to put a proposal together.
## Contact
For any questions about CNI, please reach out on the mailing list:
- Email: [cni-dev](https://groups.google.com/forum/#!forum/cni-dev)
- IRC: #[containernetworking](irc://irc.freenode.org:6667/#containernetworking) channel on freenode.org
- Slack: [containernetworking.slack.com](https://cryptic-tundra-43194.herokuapp.com)
# Release process
## Resulting artifacts
Creating a new release produces the following artifacts:
- Binaries (stored in the `release-<TAG>` directory) :
- `cni-<PLATFORM>-<VERSION>.tgz` binaries
- `cni-<VERSION>.tgz` binary (copy of amd64 platform binary)
- `sha1`, `sha256` and `sha512` files for the above files.
## Preparing for a release
1. Releases are performed by maintainers and should usually be discussed and planned at a maintainer meeting.
- Choose the version number. It should be prefixed with `v`, e.g. `v1.2.3`
- Take a quick scan through the PRs and issues to make sure there isn't anything crucial that _must_ be in the next release.
- Create a draft of the release note
- Discuss the level of testing that's needed and create a test plan if sensible
- Check what version of `go` is used in the build container, updating it if there's a new stable release.
## Creating the release artifacts
1. Make sure you are on the master branch and don't have any local uncommitted changes.
1. Create a signed tag for the release `git tag -s $VERSION` (Ensure that GPG keys are created and added to GitHub)
1. Run the release script from the root of the repository
- `scripts/release.sh`
- The script requires Docker and ensures that a consistent environment is used.
- The artifacts will now be present in the `release-<TAG>` directory.
1. Test these binaries according to the test plan.
## Publishing the release
1. Push the tag to git `git push origin <TAG>`
1. Create a release on Github, using the tag which was just pushed.
1. Attach all the artifacts from the release directory.
1. Add the release note to the release.
1. Announce the release on at least the CNI mailing, IRC and Slack.
# CNI Roadmap
This document defines a high level roadmap for CNI development.
The list below is not complete, and we advise to get the current project state from the [milestones defined in GitHub](https://github.com/containernetworking/cni/milestones).
## CNI Milestones
### [v0.2.0](https://github.com/containernetworking/cni/milestones/v0.2.0)
* Signed release binaries
* Introduction of a testing strategy/framework
### [v0.3.0](https://github.com/containernetworking/cni/milestones/v0.3.0)
* Further increase test coverage
* Simpler default route handling in bridge plugin
* Clarify project description, documentation and contribution guidelines
### [v0.4.0](https://github.com/containernetworking/cni/milestones/v0.4.0)
* Further increase test coverage
* Simpler bridging of host interface
* Improve IPAM allocator predictability
* Allow in- and output of arbitrary K/V pairs for plugins
### [v1.0.0](https://github.com/containernetworking/cni/milestones/v1.0.0)
- Plugin composition functionality
- IPv6 support
- Stable SPEC
- Strategy and tooling for backwards compatibility
- Complete test coverage
- Integrate build artefact generation with CI
File mode changed from 100755 to 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
File mode changed from 100755 to 100644
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment