How it works?

Good web crawler first accesses root of a domain and looks for robots.txt file.

For example if robot wants to check www.example.com/welcome.html it will first check if www.example.com/robots.txt exists.

And again for example it finds:

robots.txt:

# No robots, Please
User-agent: *
Disallow: /

In above file:

User-agent: * means this section applies to all robots and
Disallow: / instructs the robot that it should not visit any pages on the site.

Note: It is important to know that robots can ignore your /robots.txt and robots.txt file is a publicly available file.

First consideration is really important to know since the robots who ignore the instructions are usually malicious.

What to put inside?

robots.txt is a plain text file. Here are few examples:

To allow all robots to visit all files:
User-agent: *
Disallow:
And opposite disallow all robots out:
User-agent: *
Disallow: /

If you need to disallow a specific agent to visit specific folder
User-agent: SpecificBot # replace the 'SpecificBot' with the actual user-agent of the bot
Disallow: /notimportant/
Above example shows also how you can put comments in the file.

In addition you can tell robots where your sitemap is located
User-agent: *
Sitemap: http://www.example.com/sitemaps/sitemap.xml

Where to put it?

The short answer: in the top-level directory of your web server.

A bit longer: it should be located after your domain name. For example www.example.com/robots.txt not www.example.com/robot_file/robots.txt

After a while you notice that timestamps in your logfiles are shifted with few hours. So what could be wrong?
Actually is really simple – most probably your timezone is not correct. To check run “date” from command line, this will show what is the time for your server.
Then you find that the timezone is not correct?!

How do you set the correct one?

Unfortunately, this is not an easy thing to figure out. Official documentation states that you can use system-config-date, but it has a bunch of dependencies (when I ran yum install system-config-date on one of my servers it asked to install 84 packages).

So is there an alternative way to do it?

All timezone files are located in /usr/share/zoneinfo. To select the appropriate named timezone for your location. For my location, Montreal, Canada, I actually have to select: America/Montreal. For you it could be different so make note of the appropriate folder and file for your timezone.

The active timezone used on your system is in the /etc/localtime file.

The default will vary depending on your server host and it depends on the value that was provided during installation.
We simply need to replace this file with the file we selected in the previous step.
Even I say replace, actually is recommended to create a link to the pertinent file rather than actually replacing the file.

Here are the steps to follow:

First, backup the existing localtime file (it is always good practice to make backups of original config files).

mv /etc/localtime /etc/localtime.bak

Next, create the link:

ln -s /usr/share/zoneinfo/ /etc/localtime

Test your change.
Run “date” from the command line, and ensure that the appropriate time, date, and timezone are reported.

Simple – sometimes is required to have map if specific host to different IP ( for example: testing ).

In general is not recommended to edit hosts file. Even some viruses are using it to map popular antivirus sites to localhost and hence deny access to them.

The file is located in %systemroot%\system32\drivers\etc\ (which for most of computers translate to C:\windows\system32\drivers\etc\)

In Windows XP and lower if you are logged as administrator you can edit the file directly, but to increase security (since most of ordinary users log as administrators) Microsoft decided to add in Windows Vista (and newer) additional layer of security so you are unable to edit the file directly.

Here is how to do it:

1. In Start menu type Notepad
2. Right click on Notepad and select run as Administrator
3. Continue as usual – edit the file and save it

Note: Some antivirus products “protect” hosts file so you might need to disable that protection before editing.

Note: For newbies the format of the file is following:

x.x.x.x FQDN

Where x.x.x.x is IP address in numeric form

and FQDN is Fully Qualified Domain Name

Example:

127.0.0.1  localhost

C:\Users\>telnet
'telnet' is not recognized as an internal or external command,
operable program or batch file.

To to enable it back follow this procedure:

1. Go to Start – > Control Panel -> Programs -> Turn Windows Features On and Off
2. Check Telnet Client and click OK (See screenshot below)
3. After a while it is ready
4. Enjoy

Here is how to use it:

1. Start from command prompt
2. Type
telnet mailhost 25
Note: Replace mailhost with your email server.

3. Type:
HELO server.com

Note1: depending on the server HELO have to be replaced with EHLO
Note2: Replace server.com with your domain

4. Type
MAIL FROM:you@server.com

Note: Again replace you@server.com with your e-mail address

5. Type

RCPT TO: recipient@server2.com

Note: Replace recipient@server2.com with recipient e-mail address
6. Type

DATA
after that optionally
SUBJECT: Your subject
then type your message
to finish place dot at new line.
.
6. To exit type
QUIT

“Your temporary license about to expire in… days”

This is usually dye to misconfiguration on the server side, but here is the trick to get additional 90 days temporary license:

WARNING: This requires editing of Windows registry. Use it on your own risk.

1. Launch regedit – Start -> Run… -> regedit
2. Go to HKLM\Software\Microsoft\MSLicensing\
3. Delete hardwareID entry.
4. Enjoy another 90 days, but not forget to plan for fixing the issue

One really good tool for the job is windows experience index.

Initially my system had overall index of 2.1:

Then I checked for updates. Fortunately there were some from Microsoft. Here is the result after updates:

As you can see my index jumped to 2.3 and my graphics performance increased to from 2.7 to 3.0.

Encouraged by this I’ve installed latest video driver from NVIDIA (for Windows 7):

The results since positive My index increased to 2.4 were not so drastic, but still an improvement. Probably this is because my card is relatively old and slow or just because NVIDIA does not have(yet) drivers tuned specifically for Windows 8…

Only time will tell

Stay tuned.

Installation went smoothly and it was really quick (did not time it, but it is faster than XP installation) with 2 reboots required.

Almost all Hardware was recognized and functional – it looks like my integrated card reader give some trouble… but will work on it.

What I had to probably start with is that the license for that system is valid until March 8, 2012 (Is it a coincidence that this is international women day ).

And most importantly you have to plan that installation really carefully.

If you need to (witch is most probably the case) keep your old OS (XP/W7) you need to:

1. Create new partition for Windows 8 – this can be done with live CD like partedmagic or similar tool (windows disk management).
2. Choose that partition as destination during windows 8 install – you need to choose Custom as install method.
3. Continue installation
4. After installed go to system properties -> advanced -> startup and recovery settings and choose previous Windows version.

This procedure will show you menu on boot so you can choose what version to boot.

But how to install it on the new computers without DVD ( NO CD’s available since Win 7)?

The simplest answer is from USB key. But the question is how to “burn” DVD image to USB key?

So here is what you need:

1. Computer that covers minimum system requirements – 1GHz+ CPU, 1/2GB RAM (32/64-bit versions).
2. USB key (4G+) Smallest 32-bit image is 2.8G
3. High speed internet – do not even think downloading via 56k modem
4. You need to download appropriate version – 32 or 64 bit
5. And finally you need a tool to “burn” – take it from here.

Next you need to:

• Install the tool

• Run the tool – Simply follow the wizard.

NOTE: your USB drive will be erased so make copy of your data before using it.

Warning: See my other post “Windows 8 installation and initial experience“  before starting installation.

• Next step is to boot from USB key and install – this depends on your hardware

2-nd NOTE: This is development preview version. Do not expect it to be stable it is not intended for end users yet.