• k8s-merge-robot's avatar
    Merge pull request #24808 from screeley44/gluster_errors · 0b7f8e5b
    k8s-merge-robot authored
    Automatic merge from submit-queue
    
    read gluster log to surface glusterfs plugin errors properly in describe events
    
    glusterfs.go does not properly expose errors as all mount errors go to a log file, I propose we read the log file to expose the errors without asking the users to 'go look at this log'
    
    This PR does the following:
    1.  adds a gluster option for log-level=ERROR to remove all noise from log file
    2.  change log file name and path based on PV + Pod name - so specific per PV and Pod
    3.  create a utility to read the last two lines of the log file when failure occurs
    
    old behavior:
    ```
      13s	13s	1	{kubelet 127.0.0.1}		Warning	FailedMount	Unable to mount volumes for pod "bb-gluster-pod2_default(34b18c6b-070d-11e6-8e95-52540092b5fb)": glusterfs: mount failed: Mount failed: exit status 1
    Mounting arguments: 192.168.234.147:myVol2 /var/lib/kubelet/pods/34b18c6b-070d-11e6-8e95-52540092b5fb/volumes/kubernetes.io~glusterfs/pv-gluster glusterfs [log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/pv-gluster/glusterfs.log]
    Output: Mount failed. Please check the log file for more details.
    ```
    
    improved behavior: (updated after suggestions from community)
    ```
      34m		34m		1	{kubelet 127.0.0.1}			Warning		FailedMount	Unable to mount volumes for pod "bb-multi-pod1_default(e7d7f790-0d4b-11e6-a275-52540092b5fb)": glusterfs: mount failed: Mount failed: exit status 1
    Mounting arguments: 192.168.123.222:myVol2 /var/lib/kubelet/pods/e7d7f790-0d4b-11e6-a275-52540092b5fb/volumes/kubernetes.io~glusterfs/pv-gluster2 glusterfs [log-level=ERROR log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/pv-gluster2/bb-multi-pod1-glusterfs.log]
    Output: Mount failed. Please check the log file for more details.
    
     the following error information was pulled from the log to help resolve this issue: 
    [2016-04-28 14:21:29.109697] E [socket.c:2332:socket_connect_finish] 0-glusterfs: connection to 192.168.123.222:24007 failed (Connection timed out)
    [2016-04-28 14:21:29.109767] E [glusterfsd-mgmt.c:1819:mgmt_rpc_notify] 0-glusterfsd-mgmt: failed to connect with remote-host: 192.168.123.222 (Transport endpoint is not connected)
    
    ```
    
    also this PR is alternate approach to :  #24624 
    
    0b7f8e5b
Name
Last commit
Last update
.github Loading commit data...
Godeps Loading commit data...
api/swagger-spec Loading commit data...
build Loading commit data...
cluster Loading commit data...
cmd Loading commit data...
contrib Loading commit data...
docs Loading commit data...
examples Loading commit data...
federation Loading commit data...
hack Loading commit data...
hooks Loading commit data...
logo Loading commit data...
pkg Loading commit data...
plugin Loading commit data...
release Loading commit data...
test Loading commit data...
third_party Loading commit data...
vendor Loading commit data...
www Loading commit data...
.generated_docs Loading commit data...
.gitignore Loading commit data...
.travis.yml Loading commit data...
CHANGELOG.md Loading commit data...
CONTRIB.md Loading commit data...
CONTRIBUTING.md Loading commit data...
DESIGN.md Loading commit data...
LICENSE Loading commit data...
Makefile Loading commit data...
README.md Loading commit data...
Vagrantfile Loading commit data...
code-of-conduct.md Loading commit data...