Wednesday, May 30, 2007

useful posts at verizon issues of mailing

http://lists.alioth.debian.org/pipermail/pkg-exim4-users/2006-July/000714.html

Tuesday, May 29, 2007

Email for the single user in Debian


This is a simple tutorial aimed at those wishing to setup their Debian box to work with their ISP email account. Please send any corrections or comments to nlativy at gmail dot com. Right now on with the tutorial...

The first thing to do is get the needed packages which can be accomplished by running:

# apt-get install exim4 fetchmail mutt procmail cron vim

Of these packages you probably already have cron and exim4 also you may have noticed I slipped in vim which is unessential as you can use any other editor if you want, although why would you ;-).

Exim4

Hopefully you should have already set this up when you installed Debian, if not you can simply run:

# dpkg-reconfigure --priority=medium exim4-config

and choose "mail sent by smarthost; recieved via SMTP or Fetchmail" and set the smarthost to whatever the SMTP server of your ISP is. Note here that we are telling dpkg-reconfigure to only ask us medium priority questions and deal with the rest automatically.

Next, for exim to put the correct From: line on outgoing mail we need to add the following line to the /etc/email-addresses file:

user: someone@isp.com

substituting in your local username and your email address of course.

For myisp you need to edit
1
) /etc/exim4/email-addresses
(a symlink to /etc/email-addresses
which had a single line
loginname: loginuser@myisp.net

2) /etc/exim4/passwd.client
which had a line
outgoing.myisp.net:myisplogin:myisppasswd

3) /etc/exim4/exim4.conf.template
where we set
AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS='true'



Procmail

Procmail is a program for filtering mail, especially useful if you are subscribed to mailing lists. Setting up Procmail is pretty simple, just put the following in the ~/.procmailrc file:

MAILDIR=$HOME/mail/
LOGFILE=$HOME/.procmaillog
VERBOSE=no

# Mailing lists

# debian-user
:0
* ^TO_debian-user
debian-user/

# All other mail goes to inbox
:0
inbox/

Note the trailing "/" on the mail boxes, this tells Procmail to use the maildir format. Also now would be a good time to create the maildirs we are going to use. Simply run the following commands:

$ mkdir -p ~/mail/inbox/{cur,new,tmp}
$ mkdir -p ~/mail/sent/{cur,new,tmp}
$ mkdir -p ~/mail/debian-user/{cur,new,tmp}

You can obviously skip the last command if you aren't subscribed to debian-user.

Fetchmail

Since we are only talking about a single user the simplest way to configure fetchmail will be to run it as the user who you are collecting mail for, so in the ~/.fetchmailrc file put the following:

poll pop.isp.com with protocol pop3,
user isp_username there is local_username here,
with password isp_password;

mda '/usr/bin/procmail -f fetchmail'

replacing pop.isp.com, isp_username, local_username and isp_password with the appropriate values. Notice we are passing the mail on to procmail. Also if you use a protocol other that pop3 (which I recommend you do since POP3 sends your password over the network unencrypted) or your ISP supports some form of authentication on top of POP3 then you need to simply replace pop3 with the desired protocol -- see the fetchmail man page for a complete list of supported protocols.

Now you can test if it is all working by running:

$ fetchmail -vk

I would recommend the -v and -k flags first time in case there is a problem v makes it run verbosely and k tells it to leave the messaged on the server. Of course running fetchmail manually is a pain so we can add a cron job for it: run crontab -e, which will launch your default editor, and enter the text:

*/10 * * * * fetchmail -s

this will run fetchmail every 10 minutes and check for any new mail. The -s flag tells fetchmail to run silently (unless something goes wrong) and if you miss it out you will be emailed fetchmail's output every 10 minutes which is obviously undesirable.

What to do if your connection isn't always on

Unfortunately on my laptop the '-s' flag isn't sufficient as when I'm not on the network I get a mail every 10 minutes about fetchmail not being able to connect. To combat this I rustled up the following bash script:

#!/bin/bash

DEVICES='wlan0:eth0'
FETCHMAIL='/usr/bin/fetchmail'
OPTIONS='-s'

devices=`echo $DEVICES | sed 's/:/\\\|/g'`
ifstate=`grep '\('$devices'\)' /etc/network/ifstate`
if [[ $ifstate != "" ]] ; then
$FETCHMAIL $OPTIONS
fi

Now I run this bash script in my cron job instead of fetchmail itself and no longer get error messages just because the network is down. To save you copying and pasting this you can download the script then simply put it somewhere convenient (I use ~/bin for all my scripts) make it executable (chmod 700 whatever-you-named-the-script) and replace fetchmail -s in the above cron job with the path to wherever you put the script on your machine, for example I have:

*/10 * * * * /home/nrl/bin/fetchmail

Mutt

Finally we can setup mutt, this step should be pretty simple as most of the work is done by the aforementioned programs. All we really need is the following in the ~/.muttrc file (the first five lines are the only really essential ones):

set mbox_type=maildir
set mbox="~/mail/inbox/"
set spoolfile="~/mail/inbox/"
set folder="~/mail/"
set record="~/mail/sent/"

# Add an item for each mailbox
mailboxes ~/mail/debian-user

# unessential niceties:

# Show only important stuff in the header
ignore *
unignore from resent-from reply-to x-mailer user-agent date to cc subject

# Order to display the headers in
hdr_order From: Resent-From: Reply-To: X-Mailer: User-Agent: Date: To: Cc: Subject:

# sort messages by thread
set sort=threads

# Automatically quote message in reply
set include=yes

# Set quotemark to 1 byte
set indent_str="> "

# Only show the body when I edit a message
unset edit_headers

# for when you just can't wait for the cron job :)
# set up mutt so i can run fetchmail at any time by pressing G
macro index G "!fetchmail\n" "Invoke fetchmail"
macro pager G "!fetchmail\n" "Invoke fetchmail"

# tell mutt about my mailing lists
subscribe debian-user

I also noticed that in mutt (or at least the Debian package at time of writing) the default display of the list of messages will show the list name rather than the author when the message came from a mailing list. This is perhaps useful if all your mail is together but since we have filtered it into different maildirs it would be more useful to see the author of the message. So if you wish you can add the following line to your ~/.muttrc

set index_format="%4C %Z %{%b %d} %-15.15F (%?l?%4l&%4c?) %s"

Another thing worth noting is that we did not set your real name in the ~/.muttrc file, this is so that mutt will pull it from the /etc/passwd file. For this to work you must have set your real name when prompted during the Debian install process, if you did not do this you should now run the chfn command and set your real name as you would like it to appear on all outgoing email.

Vim

You can set vim up to run a few commands when it recognises that it is editing a mail message by adding the following to your ~/.vimrc:

augroup mail
autocmd!
autocmd FileType mail set textwidth=70 wrap nonumber
autocmd FileType mail :nmap :w:!aspell -e -c %:e
augroup END

This turns on line wrapping and sets the width of lines to 70 characters, it also turns off line numbering if you have it enabled elsewhere. The fourth line sets up vim so that you can spell check a message by simply pressing F8. This, of course, requires that aspell is installed. Note: you may prefer vimspell (see below) which offers "MS Word like" highlighting of spelling errors as you type.

muttprint, abook and urlview

What follows is unessential but thoroughly recommended for a nice mutt setup. First let's install some more programs:

# apt-get install muttprint urlview abook

Now add the following to your ~/.muttrc file:

set query_command="abook --mutt-query '%s'"
set print_command="muttprint"
macro index \cb "|urlview\n"
macro pager \cb "|urlview\n"
macro index a "|abook --add-email\n" 'add sender to abook'
macro pager a "|abook --add-email\n" 'add sender to abook'

Now mutt will run muttprint when you press "p" which will print the message nicely formatted, run abook when you press "Q" to query you address book, add the sender of the current message to your address book by pressing "a" and run urlview so you can easily view URLs in a message when you press C-b (control and b). You can also invoke abook from the command line (with the abook command surprisingly enough) to edit your address book in more detail.

Attachments and HTML mail

Another thing you need to deal with these days is the inevitable arrival of an MS Word document, pdf file or worst of all an HTML email in your inbox. Don't panic though to deal with these we can simply add the following lines to the ~/.mailcap file:

application/msword; /usr/bin/antiword '%s'; copiousoutput; description="Microsoft Word Text"; nametemplate=%s.doc
application/pdf; /usr/bin/pdftotext '%s' -; copiousoutput; description="PDF File"; nametemplate=%s.pdf
text/html; /usr/bin/lynx -force_html '%s'; needsterminal; description=HTML Text; nametemplate=%s.html

This will open up MS Word documents with the handy Antiword program so that you can have a quick look at them without having to wait for a bulky office suite to load up. The remaining two lines will open up pdf files with pdftotext (although you may prefer xpdf for this if you run mutt in X Windows) and use lynx to view HTML email. If you don't have these programs installed then you will need to run:

# apt-get install antiword lynx xpdf

Note that the pdftotext program is provided by the xpdf package.

Further reading

Now that you have mutt set up you are no doubt itching for more information so let me recommend the following:
Fighting spam on Debian with SpamAssassin - this should be the very next page you visit!
Mutt homepage - a good place to find lots of mutt related info
My first Mutt - a very nice page for new Mutt users
Mutt and GnuPG howto - a great guide to getting mutt to work with GnuPG
Exim homepage - home of all things exim :-)
Fetchmail homepage - The home of fetchmail
vimspell - a useful script for spell checking in vim
Antiword - quickly read word documents rather than waiting for an office suite to load.

Also if you are stuck don't forget to look at the man pages and you can also look at my ~/.muttrc file and my ~/.procmailrc file.


Copyright (C) 2004 Nicholas Lativy
Contributors: Florian Schlichting
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the licence can be found here.

uuid instead of /dev/sda1

Sometimes there can be problems identifying some device with install and moving around

use the
/sbin/blkid function to get information
/bin/blkid /dev/sda1

which returns the same uuid for each of the raid1 devices by the way

and

UUID=421535b5-e73d-1a2e-9a16-54b97d2ff4e6 none swap sw 0 3

here is greg folkerts comment on debian-users

How about posting it here?

---------------------------------------------

# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0

# /dev/hda3
UUID=456600fd-b794-4931-8703-bded8a1902bc / xfs defaults 0 1

# /dev/hda1
UUID=8a2ba95b-82d1-4905-a1ec-ceebb8cfc2b7 /boot ext3 defaults 0 2
#/dev/hda2
# /devUUID=f2986c54-13db-453a-80e3-6c75d19de15d none swap sw 0 0

# LVM Logical Volume (could use UUID but didn't)
/dev/mapper/stor-storLV /stor xfs defaults 0 0

# CD and DVD writer
/dev/hdd /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/hdc /media/cdrom1 udf,iso9660 user,noauto 0 0

# No Floppy in this machine
#/dev/ /media/floppy0 auto rw,user,noauto 0 0

#NFS Comments
void:/music /music nfs4 defaults,bg,posix,rsize=32768,wsize=32768 0 0
void:/stor /stor1 nfs4 defaults,bg,posix,rsize=32768,wsize=32768 0 0

Remaining problem:
What if you have a raid so you have 2 devices with same uuid
/dev/sda1 and /dev/sdb1 so what do you put in /etc/fstab
you would like to have
/dev/md0
if you simply put
UUID then if you took away one of the raid devices and then restored it, which would be mounted in /etc/fstab during boot?



Nvidia binary drivers and 2.6.20 and PARAVIRT

To summarize the complicated post before.

I. PARAVIRT

If you like the binary drivers from nvidia. Then
1. if you have an older processor such as a Non AM2 - amd x2
thus it does not have hardware virtualization, so why bother with the PARAVIRT
so just disable the PARAVIRT flag in the kernel config and recompile.

2. If you have a nice AM2 amd x2 processor then you can disable the kernel checks using the kludges given before, and have both paravirt and binary nvidia drivers.

II. Xorg module location update problem

The renaming of xorg to 1. something caused some breakage in location of modules. This is solved by using
 sh NVIDIA-Linux-x86-$NVIDVER-pkg1.run \
--x-module-path=`X -showDefaultModulePath 2>&1 | cut -d, -f1` \
--x-library-path=`X -showDefaultLibPath 2>&1`

to set up the correct location for the copying of the modules.

see
http://www.nvnews.net/vbulletin/showpost.php?p=1225802&postcount=2
1.0-9755 and older on xorg-server-1.3 and its release candidates

The 1.3 series of the X.org server has a bug where the version reported changed from 7.2.* to 1.*. This confuses nvidia-installer into thinking that your X server is very old and it installs the X modules in the wrong place. To work around this problem, use the following options:
# sh NVIDIA-Linux-version.run --x-module-path=`X -showDefaultModulePath 2>&1 | cut -d, -f1` --x-library-path=`X -showDefaultLibPath 2>&1`




Nvidia binary drivers and 2.6.20 and PARAVIRT

Seems to be a problem building nvidia binary drivers with the latest kernels.
see
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419943
related to a kernel change
The 2.6.20 kernels in unstable for 686 and k7 have CONFIG_PARAVIRT enabled.
This apparently redefines various operations widely used by kernel modules
(or included via inlined functions) to redirect through a paravirts_ops
table, but the paravirts_ops table is marked GPL-only. This produces
errors like:

FATAL: modpost: GPL-incompatible module nvidia.ko uses GPL-only
symbol 'paravirt_ops'

I've seen this problem with both nvidia (non-free) and openafs (free, but
under a non-GPL license -- its code predates the existence of Linux). Note
that these modules are not intentionally using anything related to paravirt
themselves; it looks like the paravirt.h header file is selectively
overriding functions that are pulled into the modules and which those
modules were previously using without problems.

This problem does not occur on AMD64.

see also new problems with sid upgrades

ron johnson:
The reason is explained:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419943

Ah. I compile my own kernel on a system and disable PARAVIRT.

said ron johnson search for updated sid cannot fnd nvidia driver

I got the same "can't find nvidia' error this morning when upgrading
Sid to xserver-xorg-core 1.3.0.0.dfsg-2. Thankfully I read this
thread and then the nvidia.com forums.

Running this from the console installs the driver in the new location.

# NVIDVER=1.0-9755
# sh NVIDIA-Linux-x86-$NVIDVER-pkg1.run \
--x-module-path=`X -showDefaultModulePath 2>&1 | cut -d, -f1` \
--x-library-path=`X -showDefaultLibPath 2>&1`

also notice that we have
/usr/lib/xorg/modules/
contains some modules
and
drivers are in
/usr/lib/xorg/modules/drivers

in my older machine it has

NVIDIA-Linux-x86_64-1.0-9746-pkg2.run

notice this from nvnews.net

http://www.nvnews.net/vbulletin/showpost.php?p=1225802&postcount=2
1.0-9755 and older on xorg-server-1.3 and its release candidates

The 1.3 series of the X.org server has a bug where the version reported changed from 7.2.* to 1.*. This confuses nvidia-installer into thinking that your X server is very old and it installs the X modules in the wrong place. To work around this problem, use the following options:
# sh NVIDIA-Linux-version.run --x-module-path=`X -showDefaultModulePath 2>&1 | cut -d, -f1` --x-library-path=`X -showDefaultLibPath 2>&1`
32-bit distributions with 64-bit kernels

Some distributions have the option of installing a 64-bit kernel for use with all 32-bit userspace programs. This configuration is not supported by the NVIDIA Linux Graphics Driver. If you try to install the 64-bit driver package on such a system, you will receive an error like the following:
./nvidia-installer: No such file or directory
If you have this configuration, use your distribution's package manager to install a 32-bit kernel and then install the 32-bit version of the NVIDIA Linux Graphics Driver.
Ron Johnson:
A 2.6.20.x kernel where PARAVIRT is *disabled* definitely works with
the nvidia.com 9755 driver.

here is debian binary nvidia graphics maintainer
http://www.khensu.org/index.php?itemid=182
http://www.khensu.org/index.php?blogid=1

Then there is this post: from Hugo Vanwoerkom:
i,

If you get this message with a nvidia closed driver install:

FATAL: modpost: GPL-incompatible module nvidia.ko uses GPL-only symbol
‘paravirt_ops’

and you are running Sid's latest Debian stock kernel, then there is a
workaround that was published in the nvidia linux forum:
http://www.nvnews.net/vbulletin/showthread.php?t=87541

I have reworked the solution for Debian based upon another forum thread:
http://www.nvnews.net/vbulletin/showthread.php?t=89844

1. Rebuild the linux-kbuild-2.6.20 .deb:
a. apt-get update
b. apt-get build-dep linux-kbuild-2.6.20
As user:
c. mkdir linux-kbuild-2.6.20-build
d. cd linux-kbuild-2.6.20-build
e. apt-get source linux-kbuild-2.6.20
The linux-kbuild-2.6.20 sources will be downloaded and unpacked
into a folder.
Cd to that folder, then:
f. change
linux-kbuild-2.6.20-build/linux-kbuild-2.6-2.6.20/scripts/mod/mod.c and
delete lines 1197+1198.
g. dpkg-buildpackage -uc -us -rfakeroot (the .deb is created)


2. Install the rebuilt linux-kbuild-2.6.20_2.6.20-1_i386.deb
3. apt-get install linux-image-2.6.20-1-
4. apt-get install linux-headers-2.6.20-1-
5. reboot into that kernel.
6. unpack nvidia-driver (using ”-x”)
7. run ./nvidia-installer -K (This will end with an unknown symbol error!)
8. In the /NVIDIA-Linux-x86-1.0--pkg1/usr/src/nv dir. run:
PARAVIRT_OPS=`grep “D paravirt_ops” /boot/System.map-2.6.20-1-k7 | colrm
9`
9. ld -m elf_i386 —defsym paravirt_ops=0x$PARAVIRT_OPS -r -o nvidia.ko
nvidia.o nvidia.mod.o
10. Install nvidia.ko (by copying the module over into
/lib/modules/2.6.20-1-/kernel/drivers/video
11. depmod -a

It looks like this issue will be around for awhile ;-)

Hugo
rom: ___Jul___
Resent-from: debian-user@lists.debian.org
Date: Fri, 04 May 2007 08:54:12 -0700 (PDT)
To: debian-user@lists.debian.org
Subject: Re: HOWTO:2.6.20-1-k7 + nvidia


Hi,
Based on the previous post in this thread I created a patched version of the
driver archive that consists of fixed version of modpost as well as fixed
version of the Makefile for the driver's kernel module. This all wrapped in
a script that performs the steps automaticaly and tries to ensure system's
compatibilty.

If someone is intereseted in this the package can be found here :

http://grizach.servebeer.com/nvpatch/NVIDIA-Linux-x86-100.14.03-pkg1-patched.run
http://grizach.servebeer.com/nvpatch/NVIDIA-Linux-x86-100.14.03-pkg1-patched.run

And for clarity the disclaimer that is included in the archive :


DISCLAIMER: First of all this is our very own solution and we can not
guarantee
that it will work on your computer. Allthough I don"t think so it
might
even blow your computer, more possibly just wipe something importnat
;-)
NVidia has also nothing to do with this modified version of the
driver
so I guess no official support can be expected from them either
last but not least this binary driver taints the kernel so no
support
is to be expected from Linus too :). The only ones that can help you
are either we or other users that are capable of understanding what
is
going on in here.

According to the NVidia license this version of the driver should be
legally ok:
"...Linux/FreeBSD Exception. Notwithstanding the foregoing terms
of Section 2.1.1, SOFTWARE designed exclusively for use on the Linux
or
FreeBSD operating systems, or other operating systems derived from
the
source code to these operating systems, may be copied and
redistributed,
provided that the binary files thereof are not modified in any way
(except for unzipping of compressed files)."
No binary files have been modified only Makefile.kbuild have been changed
The fixed version of modpost is under GPL and that is why source code is
included
in the archive too.


Cheers and thanks for the original solution from Hugo,
Jul
Because of the many questions how has this package been built and what it
does exactly I have written a small page that clarifies this. Its location
is :

http://grizach.servebeer.com/nvpatch/ http://grizach.servebeer.com/nvpatch/

Julian
**********



Using the NVidia driver on a Linux Kernel 2.6.2x with paravirtualization turned on



With this page I'd like to make a short introduction to how to get the NVidia driver running on a Linux Kernel version 2.6.20 or newer compiled with paravirtualization turned on.

Skip to the easy solution

I just added a new version of the NVidia driver that is tuned to work on a 2.6.21 kernel. That proves that the solution is compatible with newer kernel versions. If there are problem with installing the driver on a 2.6.20 kernel please let me know and I'll publish one assembled for that version.

This description is based on the post from Hugo in the Debian Users mailing list. It can be found here but below is the complete content of this post once again.

HOWTO:2.6.20-1-k7 + nvidia Click to flag this post 3 stars [3 stars] [3 stars] by Hugo Vanwoerkom May 02, 2007; 12:36pm :: Rate this Message: - Use ratings to moderate (?) Reply | Reply to Author | View Threaded | Show Only this Message Hi, If you get this message with a nvidia closed driver install: FATAL: modpost: GPL-incompatible module nvidia.ko uses GPL-only symbol ‘paravirt_ops’ and you are running Sid's latest Debian stock kernel, then there is a workaround that was published in the nvidia linux forum: http://www.nvnews.net/vbulletin/showthread.php?t=87541 I have reworked the solution for Debian based upon another forum thread: http://www.nvnews.net/vbulletin/showthread.php?t=89844 1. Rebuild the linux-kbuild-2.6.20 .deb: a. apt-get update b. apt-get build-dep linux-kbuild-2.6.20 As user: c. mkdir linux-kbuild-2.6.20-build d. cd linux-kbuild-2.6.20-build e. apt-get source linux-kbuild-2.6.20 The linux-kbuild-2.6.20 sources will be downloaded and unpacked into a folder. Cd to that folder, then: f. change linux-kbuild-2.6.20-build/linux-kbuild-2.6-2.6.20/scripts/mod/mod.c and delete lines 1197+1198. Please take care this is ONLY for 2.6.20. In 2.6.21 there is a case statement starting at approx the same location with two cases that have fatal() function calls in them. You can just delete the two-lined each fatal function call and that will do the same trick. g. dpkg-buildpackage -uc -us -rfakeroot (the .deb is created) 2. Install the rebuilt linux-kbuild-2.6.20_2.6.20-1_i386.deb 3. apt-get install linux-image-2.6.20-1- 4. apt-get install linux-headers-2.6.20-1- 5. reboot into that kernel. 6. unpack nvidia-driver (using ”-x”) 7. run ./nvidia-installer -K (This will end with an unknown symbol error!) 8. In the /NVIDIA-Linux-x86-1.0--pkg1/usr/src/nv dir. run: PARAVIRT_OPS=`grep “D paravirt_ops” /boot/System.map-2.6.20-1-k7 | colrm 9` 9. ld -m elf_i386 —defsym paravirt_ops=0x$PARAVIRT_OPS -r -o nvidia.ko nvidia.o nvidia.mod.o 10. Install nvidia.ko (by copying the module over into /lib/modules/2.6.20-1-/kernel/dricers/video) 11. depmod -a It looks like this issue will be around for awhile ;-) Hugo

I will try to give some hints about how to install the driver a bit easier manually and will present one patched version of this driver that can be installed automatically on most Debian systems and hopefully other distros based on Debian (like Ubuntu, Kubunto, Knoppix etc.).

First of all the steps needed to patch the kbuild system can not be avoided and IMHO are pretty well documented in the article. What a friend of mine noticed is that if you do this like the description suggests you are prompted again to update the package the next time you do apt-get upgrade/dist-upgrade. This can be avoid if you make a little trick. Instead of repackaging your fixed version do the following: [This practically means, do steps 1a - 1f and afterwards continue with the steps below]

1g. cd linux-kbuild-2.6.20-build/linux-kbuild-2.6-2.6.20/scripts/mod make 1h. apt-get install linux-kbuild-2.6.20 1i. cp -r linux-kbuild-2.6.20-build/linux-kbuild-2.6-2.6.20/scripts/mod \ /usr/src/linux-kbuild-2.6.20/scripts/ 2. Just skip this step and continue ahead.

I know this is a bit more unsafe but works well and will keep your fixed version as long as kbuild package stays the save version. I have seen the version is still ending with -1 this means they haven't changed it for the time the 2.6.20 kernel is out.

Next tip is a bit more tricky but actually helps you run nvidia-installer as it should be run without any further options or errors whatsoever.

1. Go to the directory where you have your unpacked driver (the one with nvidia-installer file) 2. cd usr/src/nv/ 3. open with your favourite editor Makefile.kbuild and just after line 77 that should read something like this : EXTRA_CFLAGS += -Wall -Wimplicit -Wreturn-type....bla bla bla.... add the following two lines: PARAVIRT_OPS := $(shell grep "D paravirt_ops" /boot/System.map-$(shell uname -r) | colrm 9) EXTRA_LDFLAGS := --defsym paravirt_ops=0x$(PARAVIRT_OPS) 4. change back to the directory of nvidia-installer ( cd ../../../ ) and run it ( ./nvidia-installer )

After your fix the installer should run without problems and the driver should be up and running as it always was. I think this steps should work for the 2.6.21 kernel as well because as you see we do use uname -r to get the kernel version up there.

Before I describe the easy way to accomplish this let me spill some words about why this problem occurs.

THE PROBLEM WITH THE 2.6.20 KERNEL WITH PARAVIRTUALIZATION

It is a long ongoing discussion about ways for the kernel developers to distinguish between drivers written under GPL(&Co.) and proprietary drivers (Like NVidia's and ATI's). The reason why they do this is because they can't and don't want to give support to people with problems running such drivers because they can not legally debug (and moreover fix) problems occuring in or around the closed-source drivers. That is why when the kernel encounters such driver it sets a so called tainted flag in the kernel. Which means this kernel has been marked once and forever (well until next reboot) as not-supported one. This is the first thing the kernel support team will look for when you post bug-report and will just delete it from their lists. So now back to the problem. Some functions are marked in the 2.6.20 kernel as gpl usage only because maybe they are somewhat related to tainting the kernel and the developers want to prevent bad driver use them at all (this last sentence is however my personal opinion it is not based on facts). What happens is that when paravirtualization support is turned on many functions in the kernel call implicitly paravirt_ops - function marked as gpl only. One of these functions is udelay - function that makes a program stay idle for some period of time. And the nvidia driver uses udelay (I can't imagine a driver that won't use udelay somehow) and therefore you can not compile the driver normally. What one can do however is first fix the tool from the kernel driver building utilities that checks for drivers not marked as GPL and still using the GPL only functions and secondly because of the kbuild package still not linking such function automatically to bad drivers give the appropriate options to the linker to circumvent this and make the driver know of paravirt_ops function. Luckily all kernel functions are listed in the System map found under /boot/System.map-[kernel_version]. That is what enables us to get the driver back running when we perform the described steps.



THE SOLUTION

Now for these of you that read these lines and think of linux being f***ing difficult and only for nuclear scientists I collected these steps in an easy script that checks if your system resembles well enough mine to be relatively sure that you can apply these steps and have the same result. Afterwards it just copies a fixed precompiled version of modpost (the program you actually mess with in steps 1 through 5 from Hugo's manual (replaced partially by steps 1g - 2 maybe :-) ). It then uses the already modified Makefile to start the nvidia installer and conclude the install. Of course as I stat ein the script too you can not hold me responsible for any damage or psycological problems you might receive by using this script. It is without any guarantee and with no obligations whatsoever from my side to help you (although I will gladly try to help you if i can and have time for).
PS. Well a lot of people have already tested and reported these packages to work so I guess they should be pretty safe but this doesn't invalidate my previous statement :).

The package can be downloaded from here
http://grizach.servebeer.com/nvpatch/NVIDIA-Linux-x86-100.14.06-pkg1-patched.run - tuned for 2.6.21
http://grizach.servebeer.com/nvpatch/NVIDIA-Linux-x86-100.14.03-pkg1-patched.run - tuned for 2.6.20
http://grizach.servebeer.com/nvpatch/NVIDIA-Linux-x86-1.0-9631-pkg1-patched.run - tuned for 2.6.20

These packages has been made with the makeself utility avaible here. This is the same tool used by the NVidia people to package their driver. It is free GPL program which does really great thing. I will certainly use it from now on when I need to create some self extracting and self running archives.

Last but not least some credits: First of all thanks to Hugo for describing this easy way to fix the NVidia drivers problem. To close transitively the thanks here I'd like to thank to the people from the mentioned nvnews threads. Last but not least I'd like to thank Peter Velichkov for finding this tutorial and actually testing it before I'd risk changing my kernel. Hell yes! This problem was the only reason I was sticking with a 2.6.18 kernel all the time till now (well the 2.6.19 was miserable mistake IMO [what can you expect from kernel with first message in the changelog - "please post BEFORE you go to parties and not AFTER" from Linus Torvalds :-P ]). Moreover it was his computer that first suffered the wrath of the script after I wrote it.

And just at the bottom the copy once again of the DISCLAIMER to be sure no one got me wrong and thinks I am violation something with this work. (Well in some sense we do overcome some kernel mechanisms to distinguish good from bad drivers, but come on, udelay not usable by any commercial drivers that is redicilous!)





DISCLAIMER: First of all this is our very own solution and we can not guarantee
that it will work on your computer. Although I don"t think so it might
even blow your computer, more possibly just wipe something important ;-)
NVidia has also nothing to do with this modified version of the driver
so I guess no official support can be expected from them either
last but not least this binary driver taints the kernel so no support
is to be expected from Linus too :). The only ones that can help you
are either we or other users that are capable of understanding what is
going on in here.

According to the NVidia license this version of the driver should be legally ok:
"...Linux/FreeBSD Exception. Notwithstanding the foregoing terms
of Section 2.1.1, SOFTWARE designed exclusively for use on the Linux or
FreeBSD operating systems, or other operating systems derived from the
source code to these operating systems, may be copied and redistributed,
provided that the binary files thereof are not modified in any way
(except for unzipping of compressed files)."
No binary files have been modified only Makefile.kbuild have been changed

The fixed version of modpost is under GPL and that is why source code is included
in the archive too.
Thanks for using my script!
Julian


Useful links
http://www.nvnews.net/vbulletin/showthread.php?t=89844
http://www.nvnews.net/vbulletin/showthread.php?t=87541
http://www.nvnews.net/vbulletin/showthread.php?t=90214

in particular
Hi!

For those of you Fedora Core 6 users who like to try out realtime preemption,
but are not willing to patch, compile and install a kernel from scratch, Ingo Molnar
is now maintaining approptiate kernel RPM packages. However, if you like to
use the nvidia linux graphics driver, with the most recent precompiled -rt kernel,
then you need to get around a few obstacles. Here is a step-by-step installation
instruction:


0. Make sure your installation does 3D acceleration with a standard kernel
(i.e. 2.6.19-1.2911.fc6)


1. Install the RT kernel:

wget http://people.redhat.com/mingo/realt...9.rt8.i686.rpm
wget http://people.redhat.com/mingo/realt...9.rt8.i686.rpm
rpm -i kernel-rt-2.6.20-0119.rt8.i686.rpm
rpm -i kernel-rt-devel-2.6.20-0119.rt8.i686.rpm


2. Modifiy "modpost" in order to accept non-GPL modules:

2a. Goto to the modpost directory

Code:
cd /usr/src/kernels/2.6.20-0119.rt8-i686/scripts/mod

2b. Delete the following two lines (1197-1198) in file modpost.c:

Code:
if (!mod->gpl_compatible)
check_for_gpl_usage(exp->export, basename, exp->name);

2c. Recompile modpost

Code:
gcc -o modpost modpost.c file2alias.c sumversion.c


3. Unpack, patch and compile the nvidia driver module

Code:
cd $PATH_TO_NVIDIA_DRIVER_PACKAGE
wget "http://www.nvnews.net/vbulletin/attachment.php?attachmentid=23993&d=1171124108"
sh NVIDIA-Linux-x86-1.0-9746-pkg1.run --extract-only
cd NVIDIA-Linux-x86-1.0-9746-pkg1/usr/src/nv
patch -p1 < ../../../../patch-nv-1.0-9746_realtime-preempt.txt make SYSSRC=/usr/src/kernels/2.6.20-0119.rt8-i686 module

Where $PATH_TO_NVIDIA_DRIVER_PACKAGE is the directory where you have
saved the nvidia driver package.


4. Manually define the missing symbol entry for "paravirt_ops"

Code:
PARAVIRT_OPS=`grep "D paravirt_ops" /boot/System.map-2.6.20-0119.rt8 | colrm 9`
ld -m elf_i386 --defsym paravirt_ops=0x$PARAVIRT_OPS -r -o nvidia.ko nvidia.o nvidia.mod.o


5. Install the nvidia kernel module

Code:
cp nvidia.ko /lib/modules/2.6.20-0119.rt8/kernel/drivers/video/
depmod -a 2.6.20-0119.rt8


Limitations
This instruction has only beed tested with 32-bit FC6. It may work for 64-bit,
but make sure to use "colrm 17" instead of "colrm 9" in step 4.


Feedback appreciated

Bernhard


Join Date: Jul 2005
Location: Munich
Posts: 447
Default Re: [Installation Instruction] nvidia driver with kernel-rt-2.6.20-0119.rt8 on FC6

Hi!

Just to let you know, it is not anymore necessary to patch the nvidia
kernel driver for version 1.0-9755 along with kernel 2.6.20-0119.rt8!

The situation is that nvidia replaced the problematic semaphore
synchronization method by "completions" some times ago, and the
high latency causing "wbinvd" instruction (when PAT support is
enabled) has been replaced in 2.6.19 by a fast and interruptible
sequence of "clflush" instructions.

However, you may still need to manually define missing symbols
for pre-compiled kernels (see earlier post for an example).

regards

Bernhard

hallo,

howto use this with Kernel 2.6.21 ?

modpost.c looks different:

if (!mod->gpl_compatible)
check_for_gpl_usage(exp->export, basename, exp->name);
check_for_unused(exp->export, basename, exp->name); '

what shall i remove - all 3 lines ?

i tested one time and i get this error during compile :

scripts/mod/modpost.c:1172: Warning: »check_for_gpl_usage« defined, but not used

scripts/mod/modpost.c:1197: Warning: »check_for_unused« defined, but not used

Thanks


CooSee ' Ya

My workaround was to disable paravirtualization in kernel
(paravirt_ops have EXPORT_SYMBOL_GPL in Module.symvers and I think this
couses the problem)
I've done this, more or less, that way:

1. Install linux-source-2.6.20-1-686
2. Uncompress /usr/src/linux-source-2.6.20-1-686.tar.bz2
3. delete symlink /lib/modules/2.6.20-1-686/build and make new (ln -s
/usr/src/linux-source-2.6.20 /lib/modules/2.6.20-1-686/build)
4. copy .config from headers to sources (cp
/usr/src/linux-headers-2.6.20-1-686/.config /usr/src/linux-source-2.6.20)
5. in sources dir 'make menuconfig' and disable paravirtualization in
'processor features'
6. make prepare
7. make scripts
8. now i could compile and install nvidia kernel drivers
here's a different solution using instructions from here: http://www.nvnews.net/vbulletin/showthread.php?t=87541

i did these steps (v1.0.9755):
2. Modifiy "modpost" in order to accept non-GPL modules (slightly different however: i recompiled the linux-kbuild-2.6.20 debian package)
unpack nvidia-driver (using "-x"), run ./nvidia-installer -K
4. Manually define the missing symbol entry for "paravirt_ops"
5. Install the nvidia kernel module

works.

debian statement on the issue, btw:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419943
also see the following article
http://kerneltrap.org/node/7545

Linux: KVM Paravirtualization

January 5, 2007 - 4:57pm
Submitted by Jeremy on January 5, 2007 - 4:57pm.

A new feature that will first be availble in the upcoming 2.6.20 kernel is KVM, a Kernel-based Virtual Machine. The project's webpage describes KVM as, "a full virtualization solution for Linux on x86 hardware. It consists of a loadable kernel module (kvm.ko) and a userspace component. Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc." The project's FAQ explains that the functionality requires "an x86 machine running a recent Linux kernel on an Intel processor with VT (virtualization technology) extensions, or an AMD processor with SVM extensions (also called AMD-V)." The userland aspect of KVM is a slighlty modified version of qemu, used to instantiate the virtual machine.

Ingo Molnar [interview] announced a new patch introducing paravirtualization support for KVM, outdating the KVM FAQ which in comparing KVM to Xen notes, "Xen supports both full virtualization and a technique called paravirtualization, which allows better performance for modified guests. kvm does not at present support paravirtualization." In describing his patch which is against the 2.6.20-rc3 + KVM trunk kernel, Ingo said it, "includes support for the hardware cr3-cache feature of Intel-VMX CPUs. (which speeds up context switches and TLB flushes)". He went on to add, "some aspects of the code are still a bit ad-hoc and incomplete, but the code is stable enough in my testing and i'd like to have some feedback." In a series of benchmarks, he found 2-task context switch performance to be improved by a factor of four, while "hackbench 1" showed twice as good performance, and "hackbench 5" showed a 30% improvement. His email goes on to detail how the paravirtualization works.


modifying iceweasel settings

I wanted to modify iceweasel so that it would not display ads.
I found information on using
userContent.css files for this at
http://www.floppymoose.com/
and dropped the file into
.mozilla/firefox/lotsoflettersnumbers.default/chrome/
where lotsoflettersnumbers is some random string you can find on your machine.

then i modified a setting or two and it worked fine.

on another machine, though, it did not work.

there were two directories .mozilla and .mozilla.org

so i tried

.mozilla/firefox/lostoflettersmumer.default/ and i created a chrome directory and dropped it in
no go
i also dropped it in
.mozilla/firefox/lostoflettersmumer.default/ itself but no go.

i also tried to drop it in
.mozilla.org/firefox/xxx.default/chrome
and
.mozilla.org/iceweasel/firefox/xxxx.default/chrome

as well as dropping it in
/etc/iceweasel/profile/chrome
so on that machine, i am stuck...

i tried a new user on that machine. it worked, she only had a .mozilla/firefox/xxx.default/chrome directory so it worked fine there.

so i just blew away my .mozilla directory (.mozilla.org.org) then started again and it created
the correct tree including a chrome directory and then i copied in the userContent.css file and i was done.

Monday, May 28, 2007

.xsession-errors file fills up the home directory

Problem:

.xsession-errors file fills up the home directory:

Idea here is to either
1) modify the lines in the /etc/X11/Xsession file from

exec >>"$ERRFILE" 2>&1
to
exec >"$ERRFILE" 2>&1

which will overwrite the xsession-errors file on each start of X,

(idea from google somewhere)


or else

2) follow the idea of
http://www.skolelinux.no/~klaus/newnotater/x2548.html

if we don't care about X errors
and simply replace
the error handling stanza with

**************
errfile="/dev/null"
exec > "$errfile" 2>&1
***************

Or else

3) set up log rotation of the .xsession-errors files in users directories.

Sunday, May 13, 2007

How to make a debian archive to carry around?

How to make a debian archive to carry around?

use debian-mirror-2.5 downloaded from

http://www.linuxinsight.com/debian_mirror_2.5.html

and edit the script, create a local repository (needs ftp at present.)

the can copy it to a hard drive with

rsync backup

we will talk about automating with the exchange of pass phrase later.

now we use

rsync -avz -e ssh etch user@192.168.55.200:/mnt/debian

careful about this. Know difference between this and

rsync -avz -e ssh etch/ username@192.168.1.57:/mnt/debian/


remember that scp will copy it again. so rsync is better ....

Saturday, May 12, 2007

How to create a mdadm managed soft raid device

add devices

I like the 4 port promise Sata300 Tx4 which works with the debian etch kernel 2.6.18-4-486.
It works with the sata_promise kernel module.

Then add drives ( I like 400G WD drives with 5 year warranties).
Then fdisk /dev/sda
n p 1
accept defaults
t fd
(ie linux autodetect raid defaults)
then
mdadm /dev/md0 --create --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
then it work work on this - takes some time...
Either before or after run
mkfs.ext3 /dev/md0
then can mount.
need to create the mdadm.conf file
so run
mdadm --detail --scan >>mdadm.conf

output looks like
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=3cfd16a0:6eaa64dd:0dfcf1b5:2690c9eb
then edit the file so it looks like this

DEVICE /dev/sda1 /dev/sdb1
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=3cfd16a0:6eaa64dd:0dfcf1b5:2690c9eb

then copy it to
/etc/mdadm/mdadm.conf

Now I sometimes have servers that are located in weird places and want to
insure remounting forcing even if raid is degraded.
so i dont put the mount point in /etc/fstab
and i create an init.d script to force assemble the raid
and mount it.
mdadm -A (need to check syntax here)
to be continued..

Sunday, May 6, 2007

linking to libraries

sometimes I install packages from outside debian (such as volpack needed
by amide a medical image display program).

How to get the libraries to be seen by the linker? they are in usual place
/usr/local/lib.
So idea is
Check your /etc/ld.so.conf file, and make sure /usr/local/lib is in it. If
it's not, add it, and rerun /sbin/ldconfig.

Yay! no more setting
export LD_LIBRARY_PATH=/usr/local/lib
amide