PTV 1000 and Widi Working in Windows 8.1

I recently purchased a Wireless Display Adapter, PTV 1000, from Amazon. I have a Dell Latitude 3330 laptop that comes with Intel Centrino N 6235 Wireless Card with WiDi. I chose PTV 1000 over PTV 3000 since my TV doesn't have HDMI input (pretty old model) and PTV1000 has both composite and HDMI output. My laptop came with Windows 8 but i free upgraded to Windows 8.1. Before purchasing I checked and made sure that the adapter works with Windows 8. Found that i may have to reinstall few things but that was okay with me. I had all the latest drivers installed for my Graphics Card, Wifi Card and used WiDi update tool to update my Widi version. I also had Virtualbox running in the system for my Ubuntu usage. Enough with the background.

As soon as I got the adapter I connected it to the TV and it displayed "Ready to Connect" . From the charms menu i selected Device->Project but there wasn't any option to add Wireless Display in that menu as explained here.

Then i tried suggestions from various forums and blogs given below. Before trying each suggestion i downloaded all software mentioned in those links so i will have everything required just in case if i get disconnected from Internet; I also had a copy of the latest drives from Dell so i can always install them back when required. Had also created a system restore image in an external HDD so i can recover from OS failures if i messed with anything as i explored this

Note that each Widi version I download is 200+MB and each Network driver version is 90+MB. I have downloaded atleast 5 different versions of WiDi (couldn't install some of them in Windows 8.1) and 6 different versions of Network driver (all of them worked and I was able to use Wifi but couldn't detect or connect to WiDi)

Below are the links i referred to:
http://forum1.netgear.com/showthread.php?t=82995&page=2
http://intel-openport.hosted.jivesoftware.com/message/174635
http://intel-openport.hosted.jivesoftware.com/message/174217#174217
http://forum1.netgear.com/showthread.php?s=30124944f98fc29a7352108b1bae56bd&t=52343&page=2

As i made progress the first settings that worked for me was a combination of solution in link 1 and link 2 above. Uninstall Intel Graphics, Intel Proset Wireless (no Widi App since i was in Win 8.1). Note that since i wanted to do this fast i did not reboot after i uninstalled the existing versions. But WiDi did not work. So i uninstalled all of them, as suggested in the link,again, rebooted and then
1. installed Graphics Driver (Win64_15338) and rebooted
2. Installed Intel Pro Wireless (Wireless_15.3.1_s64) and rebooted
3. Installed WiDi (Setup3.0.13.0) downloaded from Netgear and Rebooted

After this i was able to connect to the PTV adapter using Widi Tool (Charms did not work yet). It asked for password and I could see my desktop. That is it! None of the windows i opened showed up in TV - i tried both mirrored display and extended display option. Even tried to Move one window from laptop to extended TV display but it did not show up either. TV display was kind of stuck showing the initial desktop alone - i could see mouse pointer moving

Now since i was sure that the adapter is in working condition I was happy to try other options. First uninstall all - graphics, Wireless and WiDi. Tried installing other versions of Widi after installing the drivers mentioned above, most of them detected the adapter and showed the desktop. No improvement. Then installed the versions suggested in this link and modified the registry as suggested. I thought after this I should be able to see the option to connect to Wireless devices from Charms but it never showed up. Found the solution for that here - we should disable all Virtual Adapters. Went to device Manager and disabled the Virtualbox's adapter and i could see the add wireless display option - however it couldn't detect the adapter :(

By now i had installed and re-installed multiple combinations desperately trying to get this work with my Windows 8.1 machine and the maximum i could get was to view the desktop in TV and hear sound from any Application i ran (tried with vlc) - mouse pointer was visible. I even tried upgrading the Adapter firmware but it didn't work either so did factory reset and reverted back to old version. 

Finally the only option i could think of was of trying the drivers and WiDi version supplied by Dell for Windows 8 which was shipped with my laptop.

So i downloaded the Widi App v 4.0.18.0 for Windows 8 from Dell. Uninstall all running graphics, network and Widi. Rebooted the machine and cleanly installed the following:

1. installed Graphics Driver (Win64_15338) and rebooted
2. Installed Intel Pro Wireless (Wireless_16.7.0_e164) and rebooted
3. Intel_WiDi_4.0_PV_4.0.18.0_setup_ZPE and rebooted

I could connect to the Adapter and see the desktop. Tried playing a video and it worked! Hurrah! I was also able to setup the display to 1366x768 (maximum supported by my laptop) for both the displays

Below are the versions i am currently running on and everything works perfectly. Since my TV is old text is not clear but i can very well view movies along with sound in both the displays

Current versions Graphics 10.18.10.3345, Network 15.3.1.2, Proset Wireless App v15.3.1.0, Widi 4.0.18.0

Hope this helps someone looking for a solution. From what i tried, for the WiDi to work correctly in Windows 8.1 with PTV1000 we must have Proset Wireless App (driver alone won't do), Latest Graphics driver is fine (but this should also be reinstalled everytime you install a newer Wireless Driver/App and a WiDi application compatible with the network driver you have installed. The version i downloaded from Dell was for my laptop model (Dell 3330) not sure if other laptop models have different versions

Ubuntu 13.10 Beta testing issues in upgrading few packages

I upgraded to Ubuntu 13.10 Beta couple of weeks back. Dist-upgrade exited with error. I didn't give attention to the error that time. Since last week i noticed that some package upgrades failed. Trying to troubleshoot it i found that initscripts installation is failing with error - cannot create cannot create /var/log/dmesg: Is a directory. To my surprise there was directory /var/log/dmesg and under that there were some files like boot-repair.log and one directory each of the partition i had (eg: /var/log/dmesg/sda). Since this was a test setup i went ahead and moved the directory sudo mv /var/log/dmesg /var/log/dmesg.old

Then did this: sudo apt-get update && sudo apt-get dist-upgrade

Voila! all packages installed correctly. Note that while i had around 64 packages waiting to be installed (and unable to install) because of this issue the system was fairly stable and didn't face any issue operating it normally. I did not use it heavily though

Recovering Windows 8 bootloader in UEFI BIOS dual booting Ubuntu

Continuation of my last post...My experience recovering Windows 8 boot loader

Recovery attempt using Windows Installation/Recovery Disk

Since none of my previous attempts to restore Windows 8 boot loader worked, the only option i have now is to somehow recover windows 8 without losing data or my partitions. I had created a recovery point in windows but that will work only if i could somehow boot into windows using installation or recovery CD. Unfortunately i did not create recovery disk yet since this was new laptop. I also found a recovery disk here but didn't try it since i wasn't sure if it was safe to use.

I didn't have windows 8 OS media either. Downloaded it from the Internet and burned it into a DVD using the external DVD i had (my laptop doesn't have optical drive). In parallel tried to find and read as much help as i could possibly find in the Internet. Below I have back linked some of the links I got much info from.

Attempt to recover using Windows 8 installation media

Booted to Windows 8 from the DVD i just created. It did not ask for "Repair windows" option as mentioned in the tutorial here. Instead it just asked me to choose the Keyboard layout and went ahead with installing Windows 8. So i cancelled it immediately. I didn't want to install the OS again - all i wanted was to find a way to recover the existing OS. Note that if this step had showed me that windows is present in the disk and the recovery options for the same it would have been easy.

Attempt to recover using Windows 8 Recovery disk

I had a Windows 8 recovery Disk from another computer (HP laptop). I booted from the recovery CD. It showed the following options as shown here: System Restore, System Image Recovery, Automatic Repair and Command prompt). First tried the System Restore option and it threw error saying it couldn't find any image. Strange since i had created an image. Entered into command prompt to troubleshoot the same and found that hard disk wasn't detected. This may be because driver for the hard drive was not present in the recovery CD (may be because the recovery disk I used was for another laptop). I had all drivers in another partition in my laptop. So booted to Ubuntu, copied the drivers to an external USB drive. Rebooted using recovery CD again. Went into command prompt and loaded the drivers for my harddrive using drvload command. I was able to see the drive and all partitions in the drives from command prompt. Read this for more help on listing drives from command prompt.

Attempt 1:

I could see my windows partitions. So proceeded with boot loader recovery.

bootrec /fixmbr
bootrec /fixboot

Rebooted. Still couldn't boot into windows (not sure why!). I felt that using recovery disk of another laptop might be risky but i am not sure if that is true. But now that i know the problem we saw above (no recovery option shown) was due to driver issue, may be i can load the drivers manually and attempt other options. So tried that. 

At the installation prompt Hit Shift+F10 and used drvload to load drivers. As soon as i came out of command prompt it showed me the Recover Windows Option. 

Selected Recover Windows option (had to load the drivers manually again). Then tried few more bootrec commands in command prompt:

bootrec /fixmbr
bootrec /fixboot

bootrec /scanos

bootrec /rebuildbcd

Rebooted. Still couldn't boot into windows; all it showed was a blank screen and i still don't have a clue why it didn't work. I could sense where this is going (reset?) but continued.

Note, for all the below trials I booted from Windows 8 installation media, Loaded drivers manually using Shift+F12 and Drvload and choose Recover Windows Option. Since this was a generic Windows 8 installation media i thought it would be safe to try other Recovery options. 

Trial 1:

First i tried Automatic Repair. It ran but after that too still i couldn't boot to windows 8. 

Trial 2:

Tried the System Restore option. It showed me the recovery point which was created when i installed EasyBCD. It said EasyBCD won't be available if i use that image since that was what i wanted i proceeded with that option. After 20 mins or so restore completed. 

Rebooted the machine and boom blank screen! May be this option wouldn't restore bootfiles or bootrec - so i thought.

Trial 3: Success! At last

Tried the next option Refresh Windows. This asked me to select the drive. I selected c: (thankfully all my data were in other paritions). I started to "refresh" windows again. After about 30 mins i was able to boot to Windows but this time Windows 8 boot loader showed me 3 windows 8 options all pointing to the same copy. Booted into windows. In my desktop there was a file with the list of all programs removed from the system. It for some reason didn't have many of them like Office 2013, Firefox etc. Since my data partition had all programs downloaded already i reinstalled all required apps from that. Only thing i had to redownload was Office 2013 since it installs from Office 365 servers directly (One Click Install file).

So after a 20Hrs marathon fix I got back Windows 8 and access to outlook (i had a copy of my ost file so didn't have to download all mails again).

My experience recovering Windows 8 boot loader

This is in continuation of my experience trying to install Windows 8 with Ubuntu and my EasyBCD2.2 installation overwrote MBR. I wasn't able to boot into Windows 8 at all; all i had was two Windows 8 options shown by EasyBCD2.2 and neither worked.

Here i am writing down all the methods i used "as is". Please be cautious when trying out these as you may end up losing everything (i had a copy of my data elsewhere so did not worry too much). At this point i was ready to explore all options to recover Windows 8 so i could learn the methods along the way and use them for future issues. There may be other easy methods or advanced commands that could be used in this situation but i am not exposed to those (I am from Linux world!)

Recovery attempt through UEFI BIOS

First thing i tried was to see if i can do something from the BIOS. I have worked on different BIOSes over the past years still UEFI was little different. Here we could define the OS booting file from BIOS directly; none of the older BIOSes i have worked with had this feature. Below are some intuitional trials done by me in BIOS.

Points to note before doing this:
1. In UEFI you can add entries for multiple OSes - each pointing to a specific EFI file from a bootable partition
2. I didn't disturb the EFI boot partition when i installed Ubuntu so i assumed all boot files will be there.
3. When i selected boot files for an entry i could see three partitions in the list but i selected the boot file only from first partition since i knew it was the EFI boot partition (usually this partition will have a EFI directory under which all boot files are stored. These settings may differ if you had customized the location or removed/altered boot partition.

Now for the actual entry in BIOS - in the bios menu under boot order option you can see the entries already present. You can choose to retain existing entries or remove them to avoid confusion. For each trial I removed the existing entry for windows and created a new entry with new name (say windows, Windows trial etc.) and selected one of the bootx64.efi/bootmgfw.efi/bootmgr.efi files from the boot disk (Disk0 in my case). In earlier BIOSes if bootloader/MBR is corrupted you cannot boot into any OS; with UEFI to boot into second OS you can hit F12 to create temp boot menu and select the second OS. I always kept Windows as first boot option so i hit F12 and selected Ubuntu manually whenever i wanted to use Ubuntu.

Trial 1:
Tried to reset the BIOS to defaults, recreated the Boot entries for Windows 8 and Ubuntu. Ubuntu worked. When i selected Window 8 EasyBCD's menu showed up.

Trial 2 to N: Multiple unsuccessful trials. Couldn't keep the count :)
Hoping that EasyBCD would have kept a copy of the boot file in the boot partition, I tried creating new entries for Windows 8 by selecting various bootx64.efi files available in the boot partition (under EFI/Boot, EFI/Microsoft/Boot). I tried all bootx64.efi, bootmgr.efi and bootmgfw.efi files i could find in the partition. Nothing worked; all i got was EasyBCD2.2's boot screen (that doesn't load Windows in my system)

After several tries i understood that apparently EasyBCD2.2 had overwritten windows boot files. Note I also tried removing Ubuntu entries from UEFI and recreated it. I could boot into Ubuntu, through UEFI boot menu (by hitting F12), and use it all the time. This is to ensure if the entries i put in actually work.


Recovery attempt through Ubuntu

From Ubuntu, browsing internet, i tried some suggestions and tools to fix the windows 8 boot problem. After trying several choices in boot-repair tool i gave up. This trial was a mere test to see if boot-repair would restore the original Windows 8 boot files. As I expected it didn't work though boot-repair created a boot menu with all Windows EFI files available in the boot partition (obviously none of them worked ie., displayed EasyBCD2.2 menu)

The only choice i had at this point was to boot from a Windows 8 installation CD and do a restoration of boot files by running bootfix commands.

My Windows 8 & Ubuntu dual boot Experience in UEFI BIOS

I got a new laptop few weeks back with Windows 8. Model is Dell Latitude 3330 - it is light weight has i5 processor, 8G RAM, 500G Hybrid SSD & 13.3" HD screen and HD cam - no optical drive. Performance of the laptop was superb when i first booted it. I am planning to install some apps mainly outlook 2013 (office 365 version) and some browsers like firefox, Chrome etc. My usage is very light - i use very minimal apps so i won't be installing too many apps - all i need is to check office mails and browse. I want to install Ubuntu as dual boot so have to change the partition schema.

Before installing Ubuntu i logged into Windows 8 and shrinked the main partition C: to 100G. This left me with 350+G for rest of the data. Then i booted from an external DVD drive with Ubuntu 13.04 (64Bit) and installed it. Left enough space for Ubuntu home & data partitions and also created couple of partitions for storing data - I partition drives in this way so that each partition can be handled separately. I wanted one partition to be "common" between both Windows & Ubuntu so that i can access data from both OSes when required. I also left the other partitions in the system (FACTORY reset, TOOLS) untouched.

After installing Ubuntu i rebooted the machine expecting to see an option menu but the system booted into Ubuntu directly. Boot loader menu was not displayed at all. Rebooted again and pressed F12 to BIOS boot menu. There, in the UEFI boot menu, i could see Windows and Ubuntu. I could boot into Windows and work. Then rebooted again to Linux and copied all my data to data partitions. After copying data i had 3 data partitions, 2 recovery & tools partitions & 3 Linux partitions.

Now trying to find a way to boot into Linux & Windows without hitting F12 every time I wanted a solution. Thought of trying a bootloader in Windows 8 and found this. So installed EasyBCD 2.2 in Windows 8, edited the boot options. It detected the OS automatically i had to change the option to Ubuntu though. Clicked on Write MBR. It reported success so rebooted the PC.

Shock! I had two options there. Windows 8 & Windows 8. Strange! Further none of the options took me to Windows 8. So i have completely lost the ability to boot into Windows 8. Thankfully i could hit F12 and boot into Ubuntu. All my data in other partitions were there. Started searching for solutions to recover

Completely remove Kernel and related packages in Ubuntu

I had many versions of kernel installed in my system and ran out of space in /boot. So i decided to remove old versions that i don't use anymore.

First in terminal type:
$  dpkg -l|grep Eg: dpkg -l|grep 3.5.0-16

It will typically show 4 packages:
1. linux-headers-
2. linux-headers--generic
3. linux-image--generic
4. linux-image-extra--generic

You can remove all the above 4 packages with
$sudo apt-get remove linux-headers- linux-headers--generic linux-image--generic linux-image-extra--generic

This will remove the packages, free up space and also adjust your grub automatically. Note that i am using Ubuntu 12.10 but i think this will work for old versions too.

Upgrading Ubuntu to 12.10 Beta

I am upgrading my laptop to Ubuntu 12.10 Beta1 now so i can test and report bugs. Keeping my fingers crossed to see and try all the new features

Installing Java in Ubuntu 9.04

I was trying to install the latest java version for firefox in Ubuntu 9.04. Current sun-java version available from Jaunty archives is jre1.6u14, while the latest in java site (as of now) is jre1.6u16
Go to this link: http://java.sun.com/javase/downloads/index.jsp and downloaded latest jdk as of today it is jdk-6u16.

Open terminal and cd to the download directory.

$ chmod u+x jdk-6u16-linux-i586.bin
$ sudo mkdir /usr/java
$ sudo mv jdk-6u16-linux-i586.bin /usr/java
$ cd /usr/java
$ sudo ./jdk-6u16-linux-i586.bin

Accept the license agreement and proceed installing. This will install java in the directory /usr/java/jdk1.6.0_16/

Now we have to point the system to new installed path. Do this:

$ sudo update-alternatives --install /usr/bin/java java /usr/java/jdk1.6.0_16/jre/bin/java 2;$ sudo update-alternatives --config java 

Output will be:

There are 3 alternatives which provide `java'.

Selection Alternative
------------------------------ -----------------
+ 1 /usr/lib/jvm/java-6-sun/jre/bin/java
2 /usr/java/jre1.6.0_15/bin/java
* 3 /usr/java/jdk1.6.0_16/jre/bin/java

Press enter to keep the default[*], or type selection number:

I selected 3. Now we will have to point the new java plugin path in firefox.

$ cd /usr/lib/firefox-addons/plugins/ ; sudo ln -s /usr/java/jdk1.6.0_16/jre/plugin/i386/ns7/libjavaplugin_oji.so;$ cd /usr/lib/firefox/plugins/ ; sudo ln -s /usr/java/jdk1.6.0_16/jre/plugin/i386/ns7/libjavaplugin_oji.so

Note that firefox saw the latest plugin only after i linked it to both the directories. Then the new version started working.

Installing USB modem ZTE AC2726 in Ubuntu 9.04 Jaunty

Recently i successfully installed ZTE AC-2726 modem that came with reliance wireless broadband connection. Below are the steps:

Insert the modem in USB interface and see if that gets detected.

$ lsusb
Bus 005 Device 002: ID 19d2:fff5

I noticed in /var/log/messsages that the device is detected as usb storage. To make it detected as modem we will use usb-modeswitch package. It is a Switching tool for controlling "flip flop" USB devices.

Download usb_switch package from here and install.
$ sudo dpkg -i usb-modeswitch_1.0.2-1_i386.deb

Unplug and plugin the modem back.

$ lsusb
Bus 005 Device 004: ID 19d2:fff1 <- Notice the change in product id

$ sudo modprobe usbserial vendor=0x19d2 product=0xfff1

You can see in the logs that the modem is detected correctly now. To detect & configure the dialer config run:

$ sudo wvdialconf

Check and add the missing entries in wvdial conf as found below:
$ sudo vi /etc/wvdial.conf
Phone = #777
Password = <Your Password>
Username = <Your Login Name>
Baud = 460800
Stupid Mode = 1
New PPPD = 1
Tonline = 0

Note: Give a valid username & password in the corresponding lines above and uncomment them (Hint: remove  ";" from the beginning of the line).

$ sudo wvdial

The connection started working. Note that you can also configure gnome-ppp or kpp to connect.

Asterisk Updates

In my asterisk server i installed all updates available from the repos. After rebooting the server, all my iax2 extensions stopped working.

I also found that there is one more interface virbr0 up with ip address 192.168.122.1. I stopped dnsmasq service which was listening on this ip. Then disabled dnsmasq (chkconfig dnsmasq off). Rebooted the box. Still the interface was present.

Since there is no use of this service in my server i removed dnsmaq along with all the dependencies. Did this:
# rpm -e dnsmasq libvirt-0.3.3-14.el5_3.1.i386 NetworkManager-0.7.0-4.el5_3.i386 NetworkManager-glib-0.7.0-4.el5_3.i386 libvirt-python-0.3.3-14.el5_3.1.i386

The Virtual Interface went away but still iax2 extensions were not working. Then, found the following error in /var/log/asterisk/full
ERROR[2659] chan_iax2.c: Call rejected, CallToken Support required. If unexpected, resolve by placing address x.x.x.x in the calltokenignore list or setting user yyyy requirecalltoken=no

This is due to the security enhancement done in asterisk recently. IAX2 is now secured by call tokens which my iax2 doesn't support still. So i went ahead disabling this feature for now:

vi /etc/asterisk/iax.conf
Added calltokenoptional = 0.0.0.0/0.0.0.0 to the end. Then did: /usr/sbin/amportal restart

All iax2 extensions started working again. May be i should start looking for upgrading my clients to support call tokens for improved security.

References:
http://www.venturevoip.com/allnews.php?count=99999
http://downloads.asterisk.org/pub/security/AST-2009-006.html
Details are available in this pdf: http://svn.digium.com/svn/asterisk/branches/1.4/doc/IAX2-security.pdf

Reports work in AsteriskNow 1.5

I suppose you have already installed AsteriskNow 1.5 and you would have found that reports don't work out of the box. It always says "No data". Apparently the system is not logging the calls and we have to enable this with some additional configuration. Below are the steps for that:

Login to your asterisk server as root and type yum install asterisk-addons-mysql. Select Y when it asks for installation. Once this is done you can see that all calls that go through your server are tracked and reported. But the Admin web interface displays the following error: reporting an error "Asterisk Manager Connection Failure" (If you don't get that error then below step is not required).

To fix the error go to web interface, Choose Administrators under Basic section. Select admin user and give the password (same as what you have in amportal.conf and manager.conf files). Apply changes. That's it.

After that the system will starts tracking all the calls getting through it.

AsteriskNow 1.5 + installation and configuartion

AsteriskNow 1.5 was recently released. I already had a box with 1.0.2 version and thought of giving the new version a try. Downloaded the ISO from the site and burned it to a CD. The new version is built on CentOS 5.3. I installed the new version in the same system overwriting 1.0.2. Note that the installation did not detect the old version nor did it give me any option for upgrade. Since my server was not in production i went ahead removing all the partitions and reinstalled the new version. If you have a production box backup the configuration files or install the new version in a test system and then copy the existing setup/accounts. You have been warned.

I installed AsteriskNow with the default partitions schema ( it creates all partitions under a single LVM). Once installation is complete, remove the CD and click reboot. After the OS boots it asks to configure the following:

Authentication, Firewall, Network, System Services

Authentication - Configure any extra authentication such as ldap.
Firewall - You can enable firewall (remember to allow all required ports like sip, iax & zap) if needed.
Network - Configure you network interfaces. This config sets the IP address of all interfaces in the system. Note that eth0 alone is enabled by default. Other interfaces must be enabled manually(Edit /etc/sysconfig/network-interfaces/ifcfg-ethx and set ONBOOT=yes).
System Services - Disabled any unwanted services.

After the above is done exit the wizard to login to the box. You will also have to restart the network service to enable all interfaces after login to the system as root. Immediately after the first login i ran a update

# yum update.

This installed a new version of kernel(2.6.18-128.1.6.el5), asterisk and dahdi modules. So i rebooted the box.(Default kernel is 2.6.18-128.1.1.el5).

After the system reboots, open the web interface in your favourite browser: http://<your-asterisk-server-ip>. When you click the FreePBX Administration link it asks you to enter the password. Asterisk 1.5 by default has two administrator user accounts with default password as specified below.
Username admin, Password - admin
Username freepbx, Password - fpbx

We will now change the default password of both admin accounts and secure the system. First let us change the admin user's password. In the web interface select Administrators tab in the Basic section. Click the admin user (in the top right) and give the new password of your choice. We are done.

Note: I did the following changes from root login in the asterisk box. The web interface prompts you to reload the config after every change you do. You can reload the config after all the changes are done.

To change freepbx user's password you have to change the entries in two files (we will backup the original files) and also change the mysql password.

# cd /etc/
# cp amportal.conf amportal.conf.orig
# vi amportal.conf
AMPDBPASS=<new password>

# cd /etc/asterisk
# cp cdr_mysql.conf cdr_mysql.conf.orig
# vi cdr_mysql.conf
password=<new password> 

Now we will change the password for mysql:

# mysql -p  (Just press enter in the password Prompt)
mysql > SET PASSWORD FOR freepbx@localhost=PASSWORD('new password'); 

The web interface reports the following errors at this point (Yes, these errors are there in the fresh installation itself).

Notice Memory Limit Changed - To fix this edit /etc/php.ini and change memory_limit=100M. (You have to restart httpd after this).

Default Asterisk Manager Password Used:

You have to change this in two files

# cd /etc/
# vi amportal.conf
AMPMGRPASS=<new password>

# cd  /etc/asterisk
# cp manager.conf manager.conf.orig
# vi manager.conf
secret = <new password> (Must be same as what you gave for AMPMGRPASS in amportal.conf). 

No email address for online update checks - Go to General Settings Tab under Basic and give an email address in the Update Email: text box. (I have given my email id here).

Could not reload FOP server - For this we will first install the newest FOP version and then setup httpd to run as asterisk user (By default httpd runs as apache user).

# cd /root
# wget http://www.asternic.org/files/op_panel-0.29.tar.gz
# cd /var/www/html/panel
# cp op_server.pl op_server.pl.orig
# cp operator_panel.swf operator_panel.swf.orig
# tar zxfv /root/op_panel-0.29.tar.gz
# cp op_panel-0.29/flash/operator_panel.swf .
# cp op_panel-0.29/op_server.pl .
# cd /var/lib/asterisk/bin/
# ./retrieve_conf

# cd /etc/httpd/conf
# cp httpd.conf httpd.conf.orig
# vi httpd.conf
Find the line User apache and change it to User asterisk. Save the file and exit.
# service httpd restart 

After this click on Apply Configuration Changes in the web interface. Then the webinterface reports about module upgrade. Goto Module admin tab and click Check for upgrade online. Click Upgrade all and then click Process. Press Confirm. All modules will upgrade automatically.

Now there won't be any errors or other warnings in the web interaface. You can create extensions, inbound/outbound settings as required.

I found fixes from the following links:
http://www.freepbx.org/forum/freepbx/users/freepbx-panel-flashing-red-and-green
http://www.asternic.org/
http://www.freepbx.org/support/documentation/faq/changing-the-asterisk-manager-password

Selectively relay emails using postfix

Several machines in my internal network run automated jobs and send out emails. All emails were getting delivered without problems and one fine day my public IP got listed in RBLs and the mail servers, using those RBLs, stopped accepting mails from mine.

I have now set my main network router to restrict outgoing port 25 so that my public ip doesn't get black listed again. This means Internal machines cannot send mails to any of the domains outside my network. I needed a mechanism to route all my emails through my external mail server which has a proper mx record. My external email server, that is outside my network, allows users to relay emails only after successful auth on port 465 (SSL).  I configured postfix for this.

Note that since Postfix does not deliver mails via port 465 i used stunnel for smtps connection between my Postfix box, acting as smtps client here, and the external mail server.

Let us first start with installing postfix. We will also see how to route emails from sendmail through postfix.

Note: In the below config example.com is the local domain of postfix box and example1.com is the external domain.

# yum install postfix

We will first configure stunnel to forward all connections to the external mail server.

 # vi /etc/stunnel/stunnel.conf
client = yes
[rev-smtps]
accept = 127.0.0.1:2525
connect = <externalmailserver>:465 

The above configuration makes stunnel to listen on localhost's port 2525 and forward everything from that port to mail server's 465. Certificates are accepted automatically.

Let us start stunnel.

# stunnel /etc/stunnel/stunnel.conf (You can put this line in /etc/rc.local so that the service starts automatically after the machine is reboots).

Now try connecting to the port '2525' with
# telnet localhost 2525

You will see the greeting from the remote SMTP server. Now we will configure postfix to authenticate with the remote email server and route all such connections through stunnel.

 # vi /etc/postfix/main.cf
smtp_sasl_auth_enable = yes  # Enables auth through ssl
smtp_sasl_security_options=
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd # specifies the password file
transport_maps = hash:/etc/postfix/transport # this will tell postfix to selectively forward or relay emails.
relay_domains = $mydestination, example1.com # list of all domains the server will accept to deliver email for.
inet_interfaces = $myhostname, localhost # Interfaces in which postfix listens on
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain, example.com # Local delivery domains
home_mailbox = Mailbox 

Save the file.

We can configure different email servers for each domain with postfix's transport file.

# vi /etc/postfix/transport
example1.com smtp:127.0.0.1:2525
any-other-domain.com smtp:127.0.0.1:2525 # You can mention any other domains here if your external email allows relaying mails.

Above config will forward all mails destined to example1.com and any-other-domain.com through the stunnel server which will inturn connect us to the external email server.

# vi /etc/postfix/sasl_passwd
[127.0.0.1]:2525 username@example1.com:password 

Note that you can give only one username:password combination for one email server (127.0.0.1:2525 in this case).

Save this file.

Create maps so that postfix reads the new configuration.

# postmap /etc/postfix/sasl_passwd
# postmap /etc/postfix/transport

In all my Internal servers i have sendmail. Now i have to tell sendmail to route emails through the postfix box. I did this:

# vi /etc/mail/mailtertable
example.com smtp:[x.x.x.x]    # substitute the ip address of your postfix server instead of x.x.x.x
example1.com smtp:[x.x.x.x]    # substitute the ip address of your postfix server instead of x.x.x.x
any-other-domain.com smtp:[x.x.x.x]    # substitute the ip address of your postfix server instead of x.x.x.x

You will also have to add mx entries for example.com in your DNS.

With the above config, this is what would happen:
1. Any mail sent from internal servers (incl the postfix server itself) to user@example.com will be delivered to postfix's corresponding user mailbox.
2. Any mail sent from internal servers (incl the postfix server itself) to user@example1.com (or any-other-domain.com) will be forwarded to the external mail server's 465 port after authenticating as user@exampe1.com user.
3. Any mail from poxfix box to user@localhost or user@hostname-of-the-postfix-box will be delivered locally.

Note that any mail sent to other domains through this postfix server will be rejected.

Tomcat Failover Setup with Apache2+modjk

Tomcat works fairly well under heavy loads. For critical services we will need some type of backup/failover mechanism. This tutorial is for those who want to configure apache to failover between two tomcat instances. When the first tomcat service fails, apache-httpd automatically routes the requests to second tomcat.

This tutorial shows how to configure failover load balancing between two tomcat instances in the same server (i have also shown how to do this for tomcat instances in two different servers). For this i am assuming you know how to install multiple tomcat instances in the same machine. This is only slightly different from load balancing between two tomcat instances as you see below in the sample workers.properties file.

#
# workers.properties
#
# Below Entry Specifies the load balance factor when used with a load balancing worker.
# Note: ----> lbfactor must be > 0,  Low lbfactor means less work done by the worker.
# worker.tomcat1.lbfactor=100

# In Unix, we use forward slashes:
ps=/

# list the workers by name

# Workers that are member of lb should not be in the list.
worker.list = failover

# ------------------------
# Tomcat1 Primary 
# ------------------------
worker.tomcat1.port=8009        # You can specify your tomcat instance's ajp port here
worker.tomcat1.host=localhost # Specify any hostname/IP here if tomcat is running in a remote server
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=1
worker.tomcat1.connection_pool_timeout=600
worker.tomcat1.socket_timeout=600
worker.tomcat1.redirect=tomcat2

# ------------------------
# Tomcat2 Failover
# ------------------------
worker.tomcat2.port=9009        # You can specify your tomcat instance's ajp port here
worker.tomcat2.host=localhost # Specify any hostname/IP here if tomcat is running in a remote server
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=1
worker.tomcat2.connection_pool_timeout=600
worker.tomcat2.socket_timeout=600
worker.tomcat2.activation=disabled # Requests are sent to the disabled worker only if first worker fails

# ------------------------
# Failover worker
# ------------------------
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# ----> If a worker dies, the load balancer will check its state
# once in a while. Until then all work is redirected to peer
# worker.
worker.failover.type=lb
worker.failover.balance_workers=tomcat1,tomcat2
worker.failover.sticky_session=False
worker.failover.sticky_session_force=False

# END workers.properties
 

Load Balancing tomcat with Apache and mod jk

Tomcat works fairly under heavy loads. But many a times we would like to distribute the requests to more than one server to minimize response time and there by improving the user experience. This tutorial shows how to configure load balancing (distribute the load) between two tomcat instances in the same system (i have also shown how to do this for tomcat instances in two different servers). For this i am assuming you know how to install multiple tomcat instances in the same machine and also know how to connect apache with tomcat using mod_jk.

All we will do here is to add a new worker to the worker.properties file we already saw in the previous tutorials with Apache httpd 2.x, tomcat 6.x and tomcat-connector 1.2.27.

Below is the content of workers.properties file.

#
#workers.properties
#
# Below Entry Specifies the load balance factor when used with a load balancing worker.
# Note: ----> lbfactor must be > 0, Low lbfactor means less work done by the worker.
# worker.tomcat1.lbfactor=100 

# In Unix, we use forward slashes:
ps=/

# list the workers by name

# Workers that are member of lb should not be in the list.
worker.list = loadbalancer   

# ------------------------
# First worker
# ------------------------
worker.tomcat1.port=8009      # Give the ajp port of the first tomcat instance here
worker.tomcat1.host=localhost # You can mention any other hostname or IP here
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=100    
worker.tomcat1.connection_pool_timeout=600
worker.tomcat1.socket_timeout=600
worker.tomcat1.recovery_options=16

# ------------------------
# Second worker
# ------------------------
worker.tomcat2.port=9009      # Give the ajp port of the first tomcat instance here
worker.tomcat2.host=localhost # You can mention any other hostname or IP here
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=100
worker.tomcat2.connection_pool_timeout=600
worker.tomcat2.socket_timeout=600
worker.tomcat2.recovery_options=16

# ------------------------
# Load Balancer worker
# ------------------------
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# ----> If a worker dies, the load balancer will check its state
#once in a while. Until then all work is redirected to peer
# worker.
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=tomcat1, tomcat2
worker.loadbalancer.sticky_session=False
worker.loadbalancer.sticky_session_force=False

# END workers.properties
 

Note: You can have as many tomcat instances(workers) as you want in the same box as long as your box can withstand the load.

Ftps with apache keys

FTPS (ftp with SSL) is different from Sftp, in that the later is a mostly part of ssh service. All permissions that a user has through SSH, like viewing files under other directories like /etc, will also be available through sftp. Using sftp one can download a copy of the /etc/passwd file. To safeguard this you have to enable chroot for sftp users and they will be restricted to their home directory. I found the configuration of chrooted sftp difficult. Since i had a ftp server running in the same box, with chrooted users, it made lot of sense to me to use ftps. I just had to add certificates' location and rest of the configuration was there after all.

In my other post i wrote about using Apache-tomcat SSL keys in Apache-httpd server. This post explains how to use apache-httpd SSL keys in vsftpd.

I assume that you have a working copy of vsftpd. If not please install and configure vsftpd as per your requirement. Here is what you have to do to enable ftps (ftp through SSL).

Store all your apache SSL keys in a location accessible to vsftpd.  I kept them under /root/vsftpd_keys/. There will be minimum two files (private and public key). In my case i had 3 private key (exported.key), public key (mydomain.com.crt) and two intermediate keys (cross_intermediate.crt, gd_intermediate.crt).

First we have to combine public and intermediate keys into one file:
# cd /root/vsftpd_keys/
# cat mydomain.com.crt cross_intermediate.crt intermediate.crt > vsftpd_domain_inter_comb3.crt

Note: 1. You have to concatenate the files in the same order specified above.
          2. It is fine to join just the domain & intermediate certs if your certificate chain doesn't need/have cross_intermediate.

Edit the vsftpd.conf file (Usually /etc/vsftpd/vsftpd.conf) and add the following to the end of the file:

userlist_enable=YES
ssl_enable=YES
allow_anon_ssl=YES
force_local_data_ssl=NO
force_local_logins_ssl=YES
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
rsa_cert_file=/root/vsftpd_keys/vsftpd_domain_inter_comb3.crt
rsa_private_key_file=/root/vsftpd_keys/exported.key
pasv_enable=YES
pasv_address= <your ip here> (Note: When the system is behind NAT firewall you have to give the firewall's NATed IP here).
pasv_min_port=1500
pasv_max_port=1505

The above configuration requires incoming ports 1500 to 1505 to be open in the firewall, if you have one.

Exporting tomcat keys to Apache httpd

Apache Tomcat SSL keys created with keytool are, by default, in der format. These keys cannot be used in Apache httpd since httpd, be default, expects the keys in pem(X509) format. Using the below steps you can export the tomcat's keys to apache httpd format and use it for apache.

I assume that you already have a working copy of tomcat with SSL. You need a copy of the tomcat's keystore file and the keystore password.

Let us start....

First copy the existing tomcat's keystore file to a new directory so that we don't break anything that is working.

# mkdir /root/SSL_export/
# cp <path/to/tomcat/keystore/file> /root/SSL_export/tomcat.keystore
# cd /root/SSL_export/
# keytool -list -keystore tomcat.keystore
Enter keystore password: 

The above command will print all the keys in the keystore.

Now we will export the key in DER format
# keytool -export -alias tomcat -keystore tomcat.keystore -file exported-der.crt
Enter keystore password: 
Output will be: Certificate stored in file

The certificate will be stored in exported-der.crt

Verify the certificate with this command:
# openssl x509 -noout -text -in exported-der.crt -inform der
Output will be: The whole certificate saying - who issued it  and other info like your company name etc.

Now Convert the key to PEM format so that apache can understand it:
# openssl x509 -out exported-pem.crt -outform pem -in exported-der.crt -inform der

The exported key will be in the file exported-pem.crt.

We have exported the public key and now are going to export the private key.

Download these files:
# wget  http://richfreedman.googlepages.com/ExportPrivateKey.java
# wget http://www.source-code.biz/snippets/java/Base64Coder.java.txt
# mv Base64Coder.java.txt Base64Coder.java
# javac ExportPrivateKey.java Base64Coder.java

Below command will save the exported private key in the file it is redirected to.
# java ExportPrivateKey tomcat.keystore tomcat G10BalPass > exported-pkcs8.key

We have to change it to RSA format so that apache can recognise it.
# openssl pkcs8 -inform PEM -nocrypt -in exported-pkcs8.key -out exported.key

Private key is now exported to the file exported.key.
Download the intermediate certificate from your certificate authority and keep it in the same location.

Now we should configure apache accordingly:
# cd /etc/httpd/conf.d/
# cp ssl.conf ssl.conf.orig
# vi ssl.conf (Change the following line only after commenting the old lines):
SSLCertificateFile /root/SSL_export/exported-pem.crt
SSLCertificateKeyFile /root/SSL_export/exported.key
SSLCertificateChainFile /root/SSL_export/<intermediate.crt>

Restart apache. You must be able to browse https://yourdomain.com/

Apache and Tomcat with modjk using tar

I have already put together a tutorial to connect Apache HTTP with Apache tomcat. In that i had installed httpd with rpm. This tutorial explains the steps to connect apache(http) with tomcat using modjk ;only that this time i am going to explain how to do everything from tar bundles. Here i am not going to explain how to install tomcat. Another tutorial on that is here: http://techsk.blogspot.com/2008/04/installing-tomcat.html.

In the below tutorial i will be installing apache in the default directory (/usr/local/apache2) you can change that location in your server to any other by adding --prefix=<your-install-directory> in the configuration step before calling make. Also i am installing Apache HTTP with SSL support.

When i get some time i will put together the steps for enabling SSL in apache and for using tomcat's SSL keys in Apache.

Here we go:

First download httpd:
# cd /usr/local/src/
# wget http://apache.mirrors.redwire.net/httpd/httpd-2.2.10.tar.gz
# wget http://www.apache.org/dist/httpd/httpd-2.2.10.tar.gz.md5
# md5sum httpd-2.2.10.tar.gz (was equal to the downloaded sum)
# tar zxfv httpd-2.2.10.tar.gz
# cd httpd-2.2.10

Below command will configure (prepare) apache installation with most of the shared modules. Type ./configure --help to see what they are
# ./configure --enable-so --enable-mods-shared=most --enable-ssl    (Hint: --prefix=</path/to/install/directory>)
# make
# make install
# cd /usr/local/src/
# wget http://www.trieuvan.com/apache/tomcat/tomcat-connectors/jk/source/jk-1.2.27/tomcat-connectors-1.2.27-src.tar.gz
# wget http://www.apache.org/dist/tomcat/tomcat-connectors/jk/source/jk-1.2.27/tomcat-connectors-1.2.27-src.tar.gz.md5
# md5sum tomcat-connectors-1.2.27-src.tar.gz (was equal to the downloaded sum).
# tar zxfv tomcat-connectors-1.2.27-src.tar.gz
# cd tomcat-connectors-1.2.27-src/native/
#  ./configure --with-apxs=/usr/local/apache2/bin/apxs
# make
# make install

That is it for the installation. Now we will configure apache for connecting it with tomcat
# cd /usr/local/apache2/conf/
# cp httpd.conf httpd.conf_18Nov08
# vi httpd.conf
Add the following code below the LoadModule section.

# Load mod_jk
# 
LoadModule jk_module modules/mod_jk.so

# Configure mod_jk
#
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info

JkMount *.jsp test   <- This must be the worker name you give in workers.properties. This will at(many) times go into your Virtual Host configuration.

Uncomment the line: Include conf/extra/httpd-ssl.conf
Save and Exit.
# cd /usr/local/apache2/conf/
# vi workers.properties (create new file and paste the below code)

# workers.properties
#

# In Unix, we use forward slashes:
ps=/

# list the workers by name

worker.list=test

# ------------------------
# First tomcat server
# ------------------------
worker.test.port=8009
worker.test.host=localhost
worker.test.type=ajp13

#
# END workers.properties
#

Save and exit.

Start Apache.
# /usr/local/apache2/bin/apachectl start

Now you will be able to browse all jsp pages through apache. Apache will silently be connecting with tomcat through modjk to serve those.

Blogged with the Flock Browser

Connecting Apache with tomcat using modjk

Apache HTTP and Tomcat are web servers meant for different purposes. While Apache HTTP can be used to serve static (html) and dynamic (cgi/perl) contents ,  tomcat is used to serve Java pages (jsp/servlets). We can link both Apache HTTP & Apache Tomcat using modjk (connector):

What can you do by linking them both?

1. Effectively serve both static (html) and dynamic (jsp/servlet) pages.
2. Load balance/Cluster between two or more tomcat services.

There may be more ....

I did the installation in CentOS 4.4 (32bit). I installed Apache http (rpm) using yum. I assume that you have already installed Tomcat.

Below is the tutorial:

# yum install httpd httpd-devel (httpd-devel is needed to install any module in httpd)
# wget http://apache.siamwebhosting.com/tomcat/tomcat-connectors/jk/source/jk-1.2.26/tomcat-connectors-1.2.26-src.tar.gz (download any latest version).
# wget http://www.apache.org/dist/tomcat/tomcat-connectors/jk/source/jk-1.2.26/tomcat-connectors-1.2.26-src.tar.gz.md5 (md5 check sum for the above).

Check if you have downloaded a genuine package
# md5sum tomcat-connectors-1.2.26-src.tar.gz (This must be equal to the sum downloaded with the second wget).

# tar zfxv tomcat-connectors-1.2.26-src.tar.gz
# cd tomcat-connectors-1.2.26-src
# cd native
# ./configure --with-apxs=/usr/sbin/apxs
# make
# make install

# cd /etc/httpd/conf/
# cp httpd.conf httpd.conf.orig (I always keep a copy of the original).
# vi httpd.conf

Add the following code below the LoadModule section.

# Load mod_jk
# 
LoadModule jk_module modules/mod_jk.so

# Configure mod_jk
#
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info

JkMount *.jsp test   <- This must be the worker name you give in workers.properties. This will at(many) times go into your Virtual Host configuration.

Save and Exit.

# cd /etc/httpd/conf/
# vi workers.properties (create new file and paste the below code)

# workers.properties
#

# In Unix, we use forward slashes:
ps=/

# list the workers by name

worker.list=test

# ------------------------
# First tomcat server
# ------------------------
worker.test.port=8009
worker.test.host=localhost
worker.test.type=ajp13

#
# END workers.properties
#

Restart apache. Now you can access tomcat through apache. Any request to jsp files will go to test-tomcat.

Blogged with the Flock Browser

After a long time!

I was too busy to update my pages here. With most of the deadlines passed through, I hope to add more articles starting next week.

You can Expect something on serving dynamic pages from apache with apache + mod_jk + tomcat setup,  failover setup for two tomcat instance with apache and mod_jk, LVM setup, Problems/errors and fixes in LVM, Striping in LVM.

I will also update my experiences about ISMS implementations in our company so that it will be helpful for others who need help.

Thanks for coming here.