2020年8月28日 星期五

[ 文章收集 ] Turn on Cassandra Authentication in Docker Container

 Source From Here

HowTo
I’ve recently been working on a project that uses a Cassandra database running in a Docker container, with this image. It works great, but it does not have authentication enabled by default. This is because unless you specify otherwise, the authenticator is set to AllowAllAuthenticator.

To enable authentication, just add this line to your Dockerfile:
  1. # Require user & pass for accessing Cassandra instance within container  
  2. RUN echo "authenticator: PasswordAuthenticator" >> /etc/cassandra/cassandra.yaml  
All this does is add a line to the cassandra.yaml file (the main config file for Cassandra) indicating that you want Cassandra to require a username and password for authentication.

Now, when you start up your Cassandra Docker container, you’ll be required to specify a username and password to access the database. The default login’s username is cassandra, and the default password is also cassandra. So, if you want to start a cqlsh session, you’ll have to execute (after starting a shell in the container, of course - see this, unless you’ve mapped the container to a host port):
# cqlsh -u cassandra -p cassandra

Extra Steps
If you have enabled password authentication, you may also want to consider following the procedure outlined here:
* Increase the replication factor of your system_auth keyspace to avoid getting locked out if your lone replica node goes down.
* Create a new superuser to replace the default cassandra one.
* Use the new superuser to demote the default one to NOSUPERUSER status and change its password from the default cassandra to something more secure.


[ 常見問題 ] Check If a File Exists Before Using It

 Source From Here

How-To
In this basic example, we check to see if a file exists before interacting with it (otherwise something’s not going to work as expected). We leverage the power of the os standard library and first use the Stat() function, which although it’s usually used to get information about a file, we’re only looking at the errors.

We can’t just check for err == nil because any number of errors could be returned so we pass it to IsNotExists() to confirm that it’s an error because the file does not exist:
  1. package main  
  2.   
  3. import (  
  4.     "fmt"  
  5.     "os"  
  6. )  
  7.   
  8. func main() {  
  9.     if fileExists("example.txt") {  
  10.         fmt.Println("Example file exists")  
  11.     } else {  
  12.         fmt.Println("Example file does not exist (or is a directory)")  
  13.     }  
  14. }  
  15.   
  16. // fileExists checks if a file exists and is not a directory before we  
  17. // try using it to prevent further errors.  
  18. func fileExists(filename string) bool {  
  19.     info, err := os.Stat(filename)  
  20.     if os.IsNotExist(err) {  
  21.         return false  
  22.     }  
  23.     return !info.IsDir()  
  24. }  


[ Python 常見問題 ] How to checkout a tag with GitPython

 Source From Here

Question
In a python script, I try to checkout a tag after cloning a git repository. I use GitPython 0.3.2.
  1. #!/usr/bin/env python  
  2. import git  
  3. g = git.Git()  
  4. g.clone("user@host:repos")  
  5. g = git.Git(repos)  
  6. g.execute(["git""checkout""tag_name"])  
With this code I have an error:
g.execute(["git", "checkout", "tag_name"])
File "/usr/lib/python2.6/site-packages/git/cmd.py", line 377, in execute
raise GitCommandError(command, status, stderr_value)
GitCommandError: 'git checkout tag_name' returned exit status 1: error: pathspec 'tag_name' did not match any file(s) known to git.

If I replace the tag name with a branch name, I have no problem. I didn't find informations in GitPython documentation. And if I try to checkout the same tag in a shell, I have non problem. Do you know how can I checkout a git tag in python ?

HowTo
Assuming you cloned the repository in 'path/to/repo', just try this:
  1. from git import Git  
  2.   
  3. g = Git('path/to/repo')  
  4. g.checkout('tag_name')  


[ 常見問題 ] DNS Not Resolving under Network [CentOS8]

 Source From Here

Question
We can't ping google.com inside Docker container in CentOS8:
$ docker exec -it traefik ping google.com
ping: bad address 'google.com


HowTo
Fast workaround
# sudo nano /etc/firewalld/firewalld.conf

in config file change
  1. FirewallBackend=nftables  
to
  1. FirewallBackend=iptables  
Save change and reload firewalld
# sudo systemctl restart firewalld.service

it should help you work with docker but I advise rollback this setting when docker provides solution.

[Linux 常見問題] Quickly create a large file on a Linux system

 Source From Here

Question
How can I quickly create a large file on a Linux (Red Hat Linuxsystem?

dd will do the job, but reading from /dev/zero and writing to the drive can take a long time when you need a file several hundreds of GBs in size for testing... If you need to do that repeatedly, the time really adds up. I don't care about the contents of the file, I just want it to be created quickly. How can this be done?

Using a sparse file won't work for this. I need the file to be allocated disk space.

How-To
dd from the other answers is a good solution, but it is slow for this purpose. In Linux (and other POSIX systems), we have fallocate, which uses the desired space without having to actually writing to it, works with most modern disk based file systems, very fast. For example:
# fallocate -l 10G gentoo_root.img

2020年8月27日 星期四

[ 常見問題 ] golang 安装 gRpc

 Source From Here

Question
安裝官方安裝命令:
# go get google.golang.org/grpc

是安裝不起的,會報:
package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc"(https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1: 443: i/o timeout)

How-To
原因是這個代碼已經轉移到 github 上面了,但是代碼裡面的包依賴還是沒有修改,還是 google.golang.org 這種,所以不能使用 go get 的方式安裝,正確的安裝方式:
$ git clone https://github.com/grpc/grpc-go.git $GOPATH/src/google.golang.org/grpc
$ git clone https://github.com/golang/net.git $GOPATH/src/golang.org/x/net
$ git clone https://github.com/golang/text.git $GOPATH/src/golang.org/x/text
$ go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
$ git clone https://github.com/google/go-genproto.git $GOPATH/src/google.golang.org/genproto
$ cd $GOPATH/src/
$ go install google.golang.org/grpc

Supplement
Package Management With Go Modules: The Pragmatic Guide

2020年8月24日 星期一

[ 常見問題 ] Private registry push fail: server gave HTTP response to HTTPS client

 Source From Here

Question
Try to pull image from private repo with below error:
$ docker pull myip:5000/cadvisor
Using default tag: latest
Error response from daemon: Get https://myip:5000/v1/_ping: http: server gave HTTP response to HTTPS client


How-To
I get helped from this post, two steps in total to solve this issue:
1. Create or modify /etc/docker/daemon.json
  1. "insecure-registries":["myregistry.example.com:5000"] }  

2. Restart docker daemon
# sudo service docker restart

I agree with @dmcgowan

The --insecure-registry=myip:5000 flag is not getting set on the daemon

but I have no idea about why it only occurred under docker version 1.12. I will keep this issue open in next three days, any comments are welcome.

2020年8月22日 星期六

[ 常見問題 ] Seaborn: How to add vertical lines to a distribution plot (sns.distplot)

 Source From Here

Quesiton
Using the examples from seaborn.pydata.org and the Python DataScience Handbook, I'm able to produce a combined distribution plot with the following snippet:
  1. import pandas as pd  
  2. import numpy as np  
  3. import seaborn as sns  
  4. import matplotlib.pyplot as plt  
  5.   
  6. # some settings  
  7. sns.set_style("darkgrid")  
  8.   
  9. # Create some data  
  10. data = np.random.multivariate_normal([00], [[52], [22]], size=2000)  
  11. data = pd.DataFrame(data, columns=['x''y'])  
  12.   
  13. # Combined distributionplot  
  14. sns.distplot(data['x'])  
  15. sns.distplot(data['y'])  


How can I combine this setup with vertical lines so that I can illustrate thresholds like this:


I know I can do it with matplotlib like here Dynamic histogram subplots with line to mark target, but I really like the simplicity of seaborn plots and would like to know if it's possible to do it more elegantly (and yes, I know that seaborn builds on top of matplotlib).

How-To
Just use matplotlib.axes.Axes.axvline:
  1. plt.axvline(2.80,0.17)  

And the same for the other line

Here instead of 0.17 you can put the maxima of your distribution using some variable such as maxx = max(data) or something similar. 2.8 is the position on the x-axis. Oh remember that the y-value has to be in between 0 and 1 where 1 is the top of the plot. You can rescale your values accordingly. Another obvious option is simply
  1. plt.plot([2.82.8], [0, max(data)])  


[Linux 文章收集] Diff 和 Patch 的簡易操作使用筆記

 Source From Here

Preface
diff patch 很方便, 在此紀錄幾個基本用法:
1. diff 產生 patch 檔
2. 將 patch 寫入
3. 移除此次 patch 的內容

Diff 和 Patch 的簡易操作使用筆記

產生檔案比較 patch 檔案
// Diff 產生 patch, 檔案, 資料夾都可以
// 或 diff -Naur dir1 dir2 > project.patch # Diff 產生 patch

# diff -Naur file1 file2 > project.patch

// 將 patch 寫入
# patch -p0 < project.patch

// 再做一次會詢問是否自動加上 -R 來移除之前 patch 寫入的(恢復回上一步的資料)
# patch -p0 < project.patch

// 移除回復修改 (同上一步驟, 此次就是確定要移除上次的 project.patch)
# patch -R -p0 < project.patch

兩個資料夾(檔案也可以使用此步驟) diff, patch 案例
先建立 dir1, dir2 (一樣內容), 修改 dir2, 然後產生 diff 後, patch 寫回 dir1.
# mkdir dir1
# touch dir1/a.txt
# touch dir1/b.txt
# mkdir dir2
# cp dir1/* dir2/

// 隨便寫
# vim dir2/a.txt
# vim dir2/b.txt

// 產生 diff patch
# diff -Naur dir1 dir2 > dir.patch

// dir1 就會寫入此 patch, 此時與 dir2 會是相同的內容
# patch -p0 < dir.patch

// 會詢問是否要移除之前 patch 寫入的, 可以先 'n' 都否決, 用下個指令恢復
# patch -p0 < dir.patch

// 移除回復修改 (同上一步驟, 此次就是確定要移除 dir.patch 的內容)
# patch -R -p0 < dir.patch

兩個檔案 diff, patch 案例
# echo "Hello John" > file1
# echo "Hello Peter.\nSee you" > file2

// -u, -U NUM, --unified[=NUM]: output NUM (default 3) lines of unified context
// -r, --recursive: recursively compare any subdirectories found
// -N, --new-file: treat absent files as empty
# diff -urN file1 file2 > f.patch
# cat f.patch
--- file1 2020-08-22 12:37:18.417081806 +0000
+++ file2 2020-08-22 12:37:28.580920038 +0000
@@ -1 +1 @@
-Hello John
+Hello Peter.\nSee you


# patch < f.patch // 輸入要 patch 的檔案位置 (假設輸入 file1, 此時 file1 = file2)
patching file file1

// Check content of file1
# cat file1
Hello Peter.\nSee you

# patch < f.patch // 輸入要 patch 的檔案位置 (假設輸入 file1, 此時會詢問是否要移除 f.patch 的內容)
patching file file1
Reversed (or previously applied) patch detected! Assume -R? [n]
 y

# cat file1
Hello John

Supplement
吉米花生醬: 原始碼 diff and patch
patch 與 diff 的簡單應用
patch 跟 diff 用法 @ 工作小錦囊 :: 隨意窩 Xuite日誌
Jserv's blog: quilt - 強大的 patch 管理工具

[ Python 常見問題 ] When using unittest.mock.patch, why is autospec not True by default?

  Source From  Here Question When you patch a function using  mock , you have the option to specify  autospec  as True: If you set  autospec...