Linux side:
Started with the initial Samba configuration file /etc/samba/smb.conf, here are a few important lines that I modified or turned on:
   workgroup = workgroup  #change workgroup to my own home network name here
   security = share # I need to include printer, and found security=user could add complexity to an occasional guest computers.
   encrypt password = YES
   samba password file =/etc/samba/smbpasswd 
Then I add my username to my samba password file
    $ smbpasswd -a myusername
Restart the samba daemon
    $ /etc/init.d/smb restart
Turn off the firewall:
    $ /etc/init.d/iptables stop
In fact, one just needs to open a few ports instead of turning off the entire firewall, but here is just a simple solution if the computers are safely isolated from the internet by a router.
windows side
Click “My  Network Places” -> “Set up a home or small office network”, and a set up window will appear. Just go through the rest steps following the question in the window. An early mistake that I’ve ever made before is when it asked “Select the statement that best describe the computer”, I chose the first one, i.e., directly connect to the network. In fact, the correct one shoud be the second, i.e., the computer is connected to another computer or gateway in my case, because indeed my computer is connected to my wireless router, which effectively is a small computer.
Finally, clicking “My  Network Places” will show the samba server. Clicking the server will prompt you a username and password, which is the username and password that you added in the previous Linux side step. 
I was also trying to set up access of my Linux machine to the Windows machine, but found XP SP2 firewall is blocking access. So I didn’t bother to try more.
Here is the final set up (output of testparm)
# Global parameters
[global]
        workgroup = HOLLYWOOD
        server string = Samba Server
        security = SHARE
        log file = /var/log/samba/%m.log
        max log size = 50
        socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
        printcap name = /etc/printcap
        dns proxy = No
        idmap uid = 16777216-33554431
        idmap gid = 16777216-33554431
        cups options = raw
[homes]
        comment = Home Directories
        read only = No
        browseable = No
[printers]
        comment = All Printers
        path = /var/spool/samba
        guest ok = Yes
        printable = Yes
        browseable = No