Fast way to delete milion files in Linux

Solutions for problem when trying to delete million files and rm command says:
argument too long

1. using combination of find and xargs

# find . -print0 | xargs -0 rm -f

2. using rsync with empty directory (so far the fastest way I've found)

# rsync -a --delete /path/to/empty-directory/ /path/to/dir-to-be-deleted/

Свързване на Samba с Windows Active Directory

Тази статия предполага, че вече имате инсталирана и настроена Microsoft Windows Active Directory и знаете как се работи с нея. Samba е инсталиран върху Oracle Linux Server release 6.5, но би трябвало да работи и за други Linux дистрибуции с минимални промени.

За да може да са активни Windows потребители и групи в Samba са нужни допълнителни настройки на файловата система.

За целта е нужно файловата система да бъде монтирана с допълнителни опции acl и user_xattr. Можем да проверим за default опции при монтиране ето така:

# tune2fs -l /dev/mapper/vg_server-lv_root |grep -i "mount opt"
Default mount options:    user_xattr acl

Ако не са настроени по default, можем да го направим с командата:

# tune2fs -o acl /dev/mapper/vg_server-lv_root
# tune2fs -o user_xattr /dev/mapper/vg_server-lv_root

 За да проверим в момента тази опция дали е активна, използваме следната команда:

# getfacl /
getfacl: Removing leading '/' from absolute path names
# file: .
# owner: root
# group: root
user::r-x
group::r-x
other::r-x

Инсталираме следните пакети: samba, krb5 (Kerberos), OpenLDAP. При Oracle Linux пакетите са както следва:

# rpm -qa|grep krb
pam_krb5-2.3.11-9.el6.x86_64
krb5-libs-1.10.3-15.el6_5.1.x86_64
krb5-libs-1.10.3-15.el6_5.1.i686
krb5-workstation-1.10.3-15.el6_5.1.x86_64

# rpm -qa|grep -i openldap
openldap-2.4.23-34.el6_5.1.i686
openldap-2.4.23-34.el6_5.1.x86_64

# rpm -qa|grep -i samba  
samba-client-3.6.9-169.el6_5.x86_64
samba-common-3.6.9-169.el6_5.x86_64
samba-winbind-clients-3.6.9-169.el6_5.x86_64
samba-3.6.9-169.el6_5.x86_64
samba-winbind-3.6.9-169.el6_5.x86_64

В другите дистрибуции имената на пакетите са аналогични.

Трябва да се уверим, че Samba е компилирана с поддръжка на LDAP, Kerberos, Winbind и Active Directory:

# smbd -b | grep LDAP
HAVE_LDAP_H
HAVE_LDAP
HAVE_LDAP_DOMAIN2HOSTLIST

# smbd -b | grep KRB
HAVE_KRB5_H
HAVE_ADDRTYPE_IN_KRB5_ADDRESS
HAVE_KRB5 # smbd -b | grep ADS
WITH_ADS
WITH_ADS

# smbd -b | grep WINBIND
WITH_WINBIND
WITH_WINBIN

Настройваме и тестваме Kerberos. Нека домейна да се казва HORIZON9.LOCAL и домейн сървъра се намира на адрес ad-dc-01.horizon9.local
Ето как трябва да излгежда /etc/krb5.conf:

[libdefaults]
 default_realm = HORIZON9.LOCAL
 dns_lookup_realm = true
 dns_lookup_kdc = true
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 HORIZON9.LOCAL = {
  kdc = AD-DC-01.HORIZON9.LOCAL
  admin_server = AD-DC-01.HORIZON9.LOCAL
  default_domain = HORIZON9.LOCAL
 }

[domain_realm]
 .horizon9.local = HORIZON9.LOCAL
 horizon9.local = HORIZON9.LOCAL

Тестваме дали работи:

# kinit Administrator@HORIZON9.LOCAL
Password for Administrator@HORIZON9.LOCAL:
#

Ако не покаже никакво съобщение, значи работи както трябва.


Настройките на Samba са както следва за примерен домейн HORIZON9 (/etc/samba/smb.conf):

[global]

 log file = /var/log/samba/log.%m
 max log size = 50
 security = ads
 netbios name = LINUX-SAMBA-AD
 realm = HORIZON9.LOCAL
 workgroup = HORIZON9
 idmap uid = 500-20000000
 idmap gid = 500-20000000
 winbind enum users = yes
 winbind enum groups = yes
 winbind use default domain = yes
 template homedir = /home/%U
 template shell = /bin/bash
 client use spnego = yes
 domain master = no

 vfs objects = acl_xattr
 map acl inherit = yes
 store dos attributes = yes

 username map = /etc/samba/user.map

 [share1]
    path = /home/samba
    comment = Share1
    public = yes
    browsable = yes
    guest ok = yes
    writable = yes

Файлът /etc/samba/user.map трябва да съдържа следното:

!root = HORIZON9\Administrator HORIZON9\Administrator

или който и да е потребител с администраторски права в домейна.

# net ads join -U Administrator
Enter Administrator's password:
Using short domain name -- HORIZON9
Joined 'LINUX-SAMBA-AD' to dns domain 'horizon9.local'
No DNS domain configured for linux-samba-ad. Unable to perform DNS Update.
DNS update failed!
#

Грешката при DNS update се поправя, като се добави на ръка запис в Windows DNS-а.

За да имат достъп програмите до новите потребители и групи от домейна, трябва да редактираме /etc/nsswitch.conf

Редовете:

passwd:     files
shadow:     files
group:      files

ги заменяме с:

passwd:     compat winbind
shadow:     files
group:      compat winbind

Рестартираме winbind и samba:

# service winbind restart
# service smb restart 

Добавяме групата 'Domain Admins' към ACL на файловата система, където ще се намират споделените директории (примерно /home/samba):

# setfacl -Rm g:'Domain Admins':rwx /home/samba
# chown -R nobody:"domain admins" /home/samba

Ако някоя от тези команди ви дава грешка, че не може да намери такава група, най-вероятно е да не се виждат потребителите/групите от домейна. Може да проверите дали потребителите и групите се виждат с помощта на следната команда:

# wbinfo -g
# wbinfo -u

Добавяме winbind в методите за автентикация в /etc/pam.d/. В Oracle Linux това се прави с командата authconfig-tui (за конзола) или authconfig-gtk (графичен интерфейс). Ето промените, които прави authconfig-tui:

# grep -nri winb /etc/pam.d/

/etc/pam.d/system-auth-ac:7:auth        sufficient    pam_winbind.so use_first_pass
/etc/pam.d/system-auth-ac:12:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/system-auth-ac:17:password    sufficient    pam_winbind.so use_authtok
/etc/pam.d/smartcard-auth-ac:10:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/password-auth-ac:7:auth        sufficient    pam_winbind.so use_first_pass
/etc/pam.d/password-auth-ac:12:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/password-auth-ac:17:password    sufficient    pam_winbind.so use_authtok
/etc/pam.d/smartcard-auth:10:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/fingerprint-auth-ac:10:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/fingerprint-auth:10:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/system-auth:7:auth        sufficient    pam_winbind.so use_first_pass
/etc/pam.d/system-auth:12:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/system-auth:17:password    sufficient    pam_winbind.so use_authtok
/etc/pam.d/password-auth:7:auth        sufficient    pam_winbind.so use_first_pass
/etc/pam.d/password-auth:12:account     [default=bad success=ok user_unknown=ignore] pam_winbind.so
/etc/pam.d/password-auth:17:password    sufficient    pam_winbind.so use_authtok

Ако всичко е наред, можете вече да си раздавате права за samba shares от windows машина. След като сте раздали права от Windows машината, може да проверите дали всичко е наред със споделената директория (/home/samba):

# getfacl /home/samba
getfacl: Removing leading '/' from absolute path names
# file: home/samba
# owner: administrator
# group: domain\040admins
user::rwx
user:root:rwx
user:nobody:rwx
group::rwx
group:domain\040users:r-x
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:domain\040users:r-x
default:group:domain\040admins:rwx
default:mask::rwx
default:other::r-x

Тук се виждат и добавените права за достъп от АD потребители и групи.

Забележка:

Навсякъде, където има нужда да се пише домейн или име на хост, пишете го с ГЛАВНИ БУКВИ. Samba е капризна и понякога нещата не се получават, защото нещо е написано с малки букви.

Ако искате да разрешите потребител guest за да могат компютри извън домейна да имат достъп до samba share, е нужно да добавите следното в /etc/samba/user.map :

!root = HORIZON9\guest HORIZON9\guest

От компютъра, който не е в домейна, при опит за достъпване на samba share ще ви поиска потребител и парола. Пишете за потребител: HORIZON9\guest без да въвеждате парола, и вече имате достъп до този share.

Qmail on OpenBSD 5.7 qmail-queue problem.

OpenBSD 5.7 default install with netqmail-1.06 installation.

The problem: When you try to send mail the qmail server returns the following error:

geroy@newsrv:~$ telnet localhost 25
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 mx1.polycomp.bg ESMTP
helo
250 mx1.polycomp.bg
mail from: test@mail.bg
250 ok
rcpt to: testovacc@gmail.com
250 ok
data
354 go ahead
test test 1 2 3
.
451 qq trouble creating files in queue (#4.3.0)

The solution: Default installation of OpenBSD 5.7 makes /var directory mounted with nosuid option. Remove it from /etc/fstab and then reboot.

# cat /etc/fstab
f292a8d8eba5b8dc.b none swap sw
f292a8d8eba5b8dc.a / ffs rw 1 1
f292a8d8eba5b8dc.k /home ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.d /tmp ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.f /usr ffs rw,nodev 1 2
f292a8d8eba5b8dc.g /usr/X11R6 ffs rw,nodev 1 2
f292a8d8eba5b8dc.h /usr/local ffs rw,nodev 1 2
f292a8d8eba5b8dc.j /usr/obj ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.i /usr/src ffs rw,nodev,nosuid 1 2
f292a8d8eba5b8dc.e /var ffs rw,nodev,nosuid 1 2

remove nosuid to look like this:

f292a8d8eba5b8dc.e /var ffs rw,nodev 1 2

Save and reboot.

Compiling and using PyNaCl on Windows 7

Step-by-step how to compile PyNaCl Python package on win32.

Requirements:
Python 2.7.x for Windows - download it from here.

libsodium-1.0.2-msvc - download precompiled binaries from here.
Microsoft Visual C++ Compiler for Python 2.7 - you can download MS C++ compiler for Pyhton from here.

This guide is based on https://github.com/pyca/pynacl/issues/100 and and is a focused reiteration of it.

Python for Windows is compiled with MSVC and because of that it is not possible to compile extensions with MinGW/MSYS. The main problem is with CFFI (Common Foreign Function Interface) and most probably if you succeed with compiling, the lib will not work (will hang if trying to use its functions)

1. Download and install Python for Windows

2. Download and install Microsoft Visual C++ Compiler for Python 2.7 

3. Install python setuptools (needed for MSVC++ Python) with pip:

C:> pip install setuptools

4. Download libsodium latest release with -msvc at the end. Unzip it in C:\work\libsodium-1.0.2-msvc

5. Go to C:\work\libsodium-1.0.2-msvc\Win32\Release\v120\dynamic and rename libsodium.lib to sodium.lib. On 64bit Windows use this dir: C:\work\libsodium-1.0.2-msvc\x64\Release\v120\dynamic

6. Download PyNaCl source and unzip it to C:\work\PyNaCl-0.3.0

7. Start the MSVC++ Python shell: Start -> All Programs -> Microsoft Visual C++ Compiler Package for Python 2.7 -> Visual C++ 2008 32-bit Command Prompt

8. Set these 3 variables in cmd prompt:

set INCLUDE=%INCLUDE%C:\work\libsodium-1.0.2-msvc\include
set LIB=%LIB%C:\work\libsodium-1.0.2-msvc\Win32\Release\v120\dynamic
set SODIUM_INSTALL=system

on 64bit Windows change:
set LIB=%LIB%C:\work\libsodium-1.0.2-msvc\Win32\Release\v120\dynamic
to:
set LIB=%LIB%C:\work\libsodium-1.0.2-msvc\x64\Release\v120\dynamic

9. Probably you will need these two files when compiling:

http://msinttypes.googlecode.com/svn/trunk/stdint.h
http://msinttypes.googlecode.com/svn/trunk/inttypes.h 

Download and copy them to: C:\work\libsodium-1.0.2-msvc\include

10. Start the building process:

C:\>cd C:\work\PyNaCl-0.3.0
C:\work\PyNaCl-0.3.0>python setup.py build

11. If everything is ok, then install it:

C:\work\PyNaCl-0.3.0>python setup.py install

12. Finally, copy the original libsodium.dll in PyNaCl install dir:

C:>copy C:\work\libsodium-1.0.2-msvc\Win32\Release\v120\dynamic\libsodium.dll C:\Python27\Lib\site-packages\PyNaCl-0.3.0-py2.7-win32.egg\nacl\_lib

Here is a test program (from doc examples https://pynacl.readthedocs.org/en/latest/public/) slightly modified:

import nacl.utils
from nacl.public import PrivateKey, Box

skbob = PrivateKey.generate()
pkbob = skbob.public_key  

skalice = PrivateKey.generate()

pkalice = skalice.public_key 

bob_box = Box(skbob, pkalice)
message = b"Kill all humans"
nonce = nacl.utils.random(Box.NONCE_SIZE)

encrypted = bob_box.encrypt(message, nonce)
print "Encrypted Message:", encrypted
alice_box = Box(skalice, pkbob)

plaintext = alice_box.decrypt(encrypted)
print "Plaintext Message:", plaintext

If it works, you should see something like:

C:\work>python nacltest.py
Encrypted Message: ₧╫fαIé├l(α▀W¬½♥↔≈‼╟  üRδD≈é☻'^∞v√oòΣls╣8,ƒ   ↓ü↓╓+ô╓è╣=§╣
Plaintext Message: Kill all humans
C:\work>

Java 1.7.x on Debian Squeeze manual install

Recently Oracle made changes in java package license and it will be no longer shipped with Debian. Debian people said that this prevents even updates for old packages to be released. For more information read here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=649594

Here is solution how to install newer java packages manually on Debian Squeeze.

Download the package and unpack it in /usr/lib/jvm/

# zcat jdk-7u71-linux-i586.tar.gz|tar xvf -
# mv jdk1.7.0_71 /usr/lib/jvm/java-1.7
# chown -R root:root /usr/lib/jvm/java-1.7/*

Now use update-alternatives script to add it to /etc/alternatives

# update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/java-1.7/bin/java" 1
# update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/java-1.7/bin/javaws" 1
# update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/java-1.7/bin/javac" 1

Now set the new java binaries as default (choose the one with the right path):

# update-alternatives --config java
# update-alternatives --config javac
# update-alternatives --config javaws

Multiboot NetBSD and Debian GNU/Linux with GRUB2

Debian GNU/Linux 8 Jessie Beta2 and NetBSD 7_BETA on the same pc.
 
Linux is on /dev/sda1
NetBSD partition is on /dev/sda4
Grub is installed on master boot record (/dev/sda).

Edit /etc/grub.d/40_custom and add the following lines:

menuentry "NetBSD 7 on sda4" {
        insmod ufs2
        insmod bsd
        set root=(hd0,4)
        chainloader (hd0,4)+1
}

Don't forget to do 'update-grub' before rebooting.

Multiboot FreeBSD 10.0 and Debian GNU/Linux using GRUB2

Debian GNU/Linux 8 Jessie Beta2 and FreeBSD 10.0 multiboot with grub2

Linux is on /dev/sda1
FreeBSD partition is on /dev/sda3 (ada0s3a - BSD style)
Grub is installed on master boot record.

Edit /etc/grub.d/40_custom and add the following lines:

menuentry "FreeBSD 10.0" --class freebsd --class bsd --class os {
insmod ufs2
insmod bsd
set root=(hd0,3)
kfreebsd /boot/kernel/kernel
kfreebsd_loadenv /boot/device.hints
set kFreeBSD.vfs.root.mountfrom=ufs:/dev/ada0s3a
set kFreeBSD.vfs.root.mountfrom.options=rw
}

Don't forget to run 'update-grub' after that and reboot.