How it works?

Good web crawler first accesses root of a domain and looks for robots.txt file.

For example if robot wants to check www.example.com/welcome.html it will first check if www.example.com/robots.txt exists.

And again for example it finds:

robots.txt:

# No robots, Please
User-agent: *
Disallow: /

In above file:

User-agent: * means this section applies to all robots and
Disallow: / instructs the robot that it should not visit any pages on the site.

Note: It is important to know that robots can ignore your /robots.txt and robots.txt file is a publicly available file.

First consideration is really important to know since the robots who ignore the instructions are usually malicious.

What to put inside?

robots.txt is a plain text file. Here are few examples:

To allow all robots to visit all files:
User-agent: *
Disallow:
And opposite disallow all robots out:
User-agent: *
Disallow: /

If you need to disallow a specific agent to visit specific folder
User-agent: SpecificBot # replace the 'SpecificBot' with the actual user-agent of the bot
Disallow: /notimportant/
Above example shows also how you can put comments in the file.

In addition you can tell robots where your sitemap is located
User-agent: *
Sitemap: http://www.example.com/sitemaps/sitemap.xml

Where to put it?

The short answer: in the top-level directory of your web server.

A bit longer: it should be located after your domain name. For example www.example.com/robots.txt not www.example.com/robot_file/robots.txt

Simple – sometimes is required to have map if specific host to different IP ( for example: testing ).

In general is not recommended to edit hosts file. Even some viruses are using it to map popular antivirus sites to localhost and hence deny access to them.

The file is located in %systemroot%\system32\drivers\etc\ (which for most of computers translate to C:\windows\system32\drivers\etc\)

In Windows XP and lower if you are logged as administrator you can edit the file directly, but to increase security (since most of ordinary users log as administrators) Microsoft decided to add in Windows Vista (and newer) additional layer of security so you are unable to edit the file directly.

Here is how to do it:

1. In Start menu type Notepad
2. Right click on Notepad and select run as Administrator
3. Continue as usual – edit the file and save it

Note: Some antivirus products “protect” hosts file so you might need to disable that protection before editing.

Note: For newbies the format of the file is following:

x.x.x.x FQDN

Where x.x.x.x is IP address in numeric form

and FQDN is Fully Qualified Domain Name

Example:

127.0.0.1  localhost

C:\Users\>telnet
'telnet' is not recognized as an internal or external command,
operable program or batch file.

To to enable it back follow this procedure:

1. Go to Start – > Control Panel -> Programs -> Turn Windows Features On and Off
2. Check Telnet Client and click OK (See screenshot below)
3. After a while it is ready
4. Enjoy

Here is how to use it:

1. Start from command prompt
2. Type
telnet mailhost 25
Note: Replace mailhost with your email server.

3. Type:
HELO server.com

Note1: depending on the server HELO have to be replaced with EHLO
Note2: Replace server.com with your domain

4. Type
MAIL FROM:you@server.com

Note: Again replace you@server.com with your e-mail address

5. Type

RCPT TO: recipient@server2.com

Note: Replace recipient@server2.com with recipient e-mail address
6. Type

DATA
after that optionally
SUBJECT: Your subject
then type your message
to finish place dot at new line.
.
6. To exit type
QUIT

IPMI stands for Intelligent Platform Management Interface standardized computer interface for administrators to manage computer system and monitor its operation.

It is supported by most major providers – Intel (of course), Dell, HP…

Installation

Installation is really simple:

yum install OpenIPMI OpenIPMI-tools
chkconfig ipmi on
service ipmi start

If you see error at last command check system log – it is possible that your hardware does not support IPMI.

Usage

To to see what you can do with it:

ipmitool help
man ipmitool

Enjoy:)

The rule was copy yesterday’s files to …

But How to determine what was the yesterdays date?

The answer was in date man page.

Here are the parts from the script related to yesterday date:

$date -d '1 day ago' +'%Y/%m/%d'
2011/06/02

on Internet I’ve found also:

date --date='yesterday'
Thu Jun  2 14:01:28 EDT 2011

And (Bonus!!) if you want to go in future

date --date='tomorrow'
Sat Jun  4 14:04:29 EDT 2011

Happy scripting…

It appears to be fairly simple process:

1. Edit sources.list

vi /etc/apt/sources.list

2. Replace distribution name (I appeared to have Lenny) with Squeeze.

%s/lenny/squeeze/g

Note: DebianVolatile has been superseded by squeeze-updates since Debian Squeeze.

To use squeeze-updates just add

deb http://mirrors.kernel.org/debian squeeze-updates main contrib

to sources.list

Then as usual:

apt-get update

apt-get dist-upgrade

In windows you just move your cursor at the top of the screen to the control bar and there you can click the button for windowed mode.

But tsclient does not have that instead you can use following keyboard combination: “Ctrl-Alt-Enter”.

Tip: To improve tsclient performance just go to performance tab and check enable bitmap caching.

Here you will find few tricks:

find ~ -type f mtime 0

Will show all files in your home folder created today.

find ~ -type f mtime 0 -iname '*.mp3'

Will narrow the search to .mp3 files only and iname is not case sensitive.

And if you want to search for files that are not belonging to you use:

find ~ ! -user ${USER}

Lot more options you can find in find man page

man find

As remedy to this you could still boot from a CD

But sometimes you even do not have CD/DVD drive.

And then you need to boot from floppy to update your BIOS.

So what to do?

Do not run to your basement looking for floppies. There is another way. You can use grub and syslinux.

First you need Linux installed (OR probably you already have it). Examples here are for Ubuntu.

Next step is to install needed software:
sudo aptitude install syslinux

Then download a floppy image:
http://www.allbootdisks.com/downloads/Disks/MS-DOS_Boot_Disk_Download47/Diskette%20Images/Dos6.22.img

Then mount the image:
sudo mount Dos6.22.img /mnt -o loop

You can remove everything except:

• autoexec.bat
• command.com
• config.sys
• himem.sys
• io.sys
• mscdex.exe
• msdos.sys

After that in case of upgrading BIOS you need to copy new bios file and flash program.

Finally we will prepare the boot image.
sudo cp /usr/lib/syslinux/memdisk /boot/
sudo cp Dos6.22.img /boot/

Reboot and on grub screen press c and boot the image using:
root (hd0,0)
kernel memdisk
initrd Dos6.22.img