Make your own Virtual Tape Library with regular disk storage

Posted in Linux

I know there’s already big companies making Virtual Tape Library (VTL) devices, but they’re expensive and I’m just looking for a cheap solution for testing. And it’s working quite well so far as a proof-of-concept for my needs.

After setting it up you should see a media changer and a basic set of tape drives using by using dmesg or lsscsi.

$ sudo dmesg | grep scsi

sr0: scsi3-mmc drive: 1x/1x xa/form2 cdda tray
sr 2:0:0:0: Attached scsi CD-ROM sr0
sd 0:0:0:0: Attached scsi generic sg0 type 0
sr 2:0:0:0: Attached scsi generic sg1 type 5
scsi3 : mhvtl: version 0.15.11 [20090328-0], opts=0×0
scsi 3:0:0:0: Medium Changer STK L700 5400 PQ: 0 ANSI: 5
scsi 3:0:0:0: Attached scsi generic sg2 type 8
scsi4 : mhvtl: version 0.15.11 [20090328-0], opts=0×0
scsi 4:0:0:0: Sequential-Access QUANTUM SDLT600 5400 PQ: 0 ANSI: 5
scsi 4:0:0:1: Sequential-Access QUANTUM SDLT600 5400 PQ: 0 ANSI: 5
scsi 4:0:0:2: Sequential-Access QUANTUM SDLT600 5400 PQ: 0 ANSI: 5
scsi 4:0:0:3: Sequential-Access IBM ULT3580-TD4 5400 PQ: 0 ANSI: 5
scsi 4:0:0:4: Sequential-Access IBM ULT3580-TD4 5400 PQ: 0 ANSI: 5
scsi 4:0:0:5: Sequential-Access IBM ULT3580-TD4 5400 PQ: 0 ANSI: 5
scsi 4:0:0:6: Sequential-Access SONY SDX-900V 5400 PQ: 0 ANSI: 5
scsi 4:0:0:7: Sequential-Access SONY SDX-900V 5400 PQ: 0 ANSI: 5
scsi 4:0:0:0: Attached scsi generic sg3 type 1
scsi 4:0:0:1: Attached scsi generic sg4 type 1
scsi 4:0:0:2: Attached scsi generic sg5 type 1
scsi 4:0:0:3: Attached scsi generic sg6 type 1
scsi 4:0:0:4: Attached scsi generic sg7 type 1
scsi 4:0:0:5: Attached scsi generic sg8 type 1
scsi 4:0:0:6: Attached scsi generic sg9 type 1
scsi 4:0:0:7: Attached scsi generic sg10 type 1
st 4:0:0:0: Attached scsi tape st0
st 4:0:0:1: Attached scsi tape st1
st 4:0:0:2: Attached scsi tape st2
st 4:0:0:3: Attached scsi tape st3
st 4:0:0:4: Attached scsi tape st4
st 4:0:0:5: Attached scsi tape st5
st 4:0:0:6: Attached scsi tape st6
st 4:0:0:7: Attached scsi tape st7

This host is a VMware 32-bit Fedora 10 installation running on VMware Server 2.0. I’ll try it on a real piece of hardware with a dedicated disk mounted where the tape data gets written to.

Start by getting the source here. I grabbed the mhvtl-2009-04-09.tgz file containing the userspace and kernel module source code.

Make the kernel module first.

$ tar xvzf mhvtl-2009-04-09.tgz

$ cd mhvtl/kernel && make

make -C /lib/modules/2.6.30-rc2/build SUBDIRS=/home/test/mhvtl-0.15/kernel modules
make[1]: Entering directory `/usr/src/kernels/2.6.30-rc2-i686'
  CC [M]  /home/test/mhvtl-0.15/kernel/mhvtl.o
  LD [M]  /home/test/mhvtl-0.15/kernel/vtl.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/test/mhvtl-0.15/kernel/vtl.mod.o
  LD [M]  /home/test/mhvtl-0.15/kernel/vtl.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.30-rc2-i686'

$ sudo make install

install -o root -g bin -m 755 vtl.ko /lib/modules/`uname -r`/kernel/drivers/scsi/

Now make the vtl user space daemons

$ cd .. && make

make -C usr
make[1]: Entering directory `/mhvtl-0.15/usr'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/mhvtl-0.15/usr'

$ cd /lib/modules`uname -r`/kernel/drivers/scsi

This might be because i’m doing this in a VMware virtual machine, but modprobe would not see the vtl.ko at all, yet insmod would load it. The scripts use modprobe, so I manually loaded it with insmod to get around this problem for now.

$ sudo insmod vtl.ko

Launch the init script.

$ sudo /etc/init.d/vtl start

Posted by admica   @   16 April 2009

Related Posts

8 Comments

Comments
May 18, 2009
12:59 pm
#1 P2k1 :

Hey
you have to exec. “depmod -a”, to get the modul “working”! ;)

May 19, 2009
8:54 pm
#2 Tom :

I love it. Worked well in NetBackup but could not get the Medium Changer to work in TSM .. Any idea ?

I get this in the SYSLOG:

May 19 22:28:21 CentOS vtllibrary[17959]: SCSI s/n: (104)
May 19 22:28:21 CentOS vtllibrary[17959]: 16 00 00 00 00 00
May 19 22:28:21 CentOS vtllibrary[17959]: completeSCSICommand: op s/n: (104), sam_status: 0, sz: 5
May 19 22:28:21 CentOS vtllibrary[17959]: SCSI s/n: (106)
May 19 22:28:21 CentOS vtllibrary[17959]: 1a 08 1d 00 18 00
May 19 22:28:21 CentOS vtllibrary[17959]: MODE SENSE **
May 19 22:28:21 CentOS vtllibrary[17959]: Mode Sense 6 byte version
May 19 22:28:21 CentOS vtllibrary[17959]: Page Control : Current configuration(0×0)
May 19 22:28:21 CentOS vtllibrary[17959]: Page Code : 0×1d
May 19 22:28:21 CentOS vtllibrary[17959]: Disable Block Descriptor => No
May 19 22:28:21 CentOS vtllibrary[17959]: Allocation len: 24
May 19 22:28:21 CentOS vtllibrary[17959]: find_pcode(0×1d): match pcode 29
May 19 22:28:21 CentOS vtllibrary[17959]: completeSCSICommand: op s/n: (106), sam_status: 0, sz: 29
May 19 22:28:21 CentOS vtllibrary[17959]: SCSI s/n: (107)
May 19 22:28:21 CentOS vtllibrary[17959]: 1a 08 1d 00 18 00
May 19 22:28:21 CentOS vtllibrary[17959]: MODE SENSE **
May 19 22:28:21 CentOS vtllibrary[17959]: Mode Sense 6 byte version
May 19 22:28:21 CentOS vtllibrary[17959]: Page Control : Current configuration(0×0)
May 19 22:28:21 CentOS vtllibrary[17959]: Page Code : 0×1d
May 19 22:28:21 CentOS vtllibrary[17959]: Disable Block Descriptor => No
May 19 22:28:21 CentOS vtllibrary[17959]: Allocation len: 24
May 19 22:28:21 CentOS vtllibrary[17959]: find_pcode(0×1d): match pcode 29
May 19 22:28:21 CentOS vtllibrary[17959]: completeSCSICommand: op s/n: (107), sam_status: 0, sz: 29
May 19 22:28:21 CentOS vtllibrary[17959]: SCSI s/n: (108)
May 19 22:28:21 CentOS vtllibrary[17959]: b8 10 00 00 00 30 00 00 10 e0 00 00
May 19 22:28:21 CentOS vtllibrary[17959]: READ ELEMENT STATUS **
May 19 22:28:21 CentOS vtllibrary[17959]: Element type(0) => All Elements
May 19 22:28:21 CentOS vtllibrary[17959]: Starting Element Address: 0
May 19 22:28:21 CentOS vtllibrary[17959]: Number of Elements : 48
May 19 22:28:21 CentOS vtllibrary[17959]: Allocation length : 4320
May 19 22:28:21 CentOS vtllibrary[17959]: Device ID: No, voltag: Yes
May 19 22:28:21 CentOS vtllibrary[17959]: completeSCSICommand: op s/n: (108), sam_status: 2, 0×05/0×24/0×00, sz: 5
May 19 22:28:21 CentOS vtllibrary[17959]: SCSI s/n: (109)
May 19 22:28:21 CentOS vtllibrary[17959]: 17 00 00 00 00 00
May 19 22:28:21 CentOS vtllibrary[17959]: completeSCSICommand: op s/n: (109), sam_status: 0, sz: 5

May 28, 2009
1:07 pm
#3 admica :

I read in the docs that the author has tested it with NetBackup but has had problems in NetWorker. (docs are under /usr/share/doc/mhvtl*) The scsi command set is only partially implemented in the user space daemons. Perhaps you can make a patch to get it to work with other software and send it to the author?

Sep 21, 2009
7:14 am
#4 Quicks :

How do I create a virtual tape library like this for a windows environment,

Sep 25, 2009
2:14 pm
#5 admica :

I’ve used CommVault backup software. It’s my favorite enterprise level backup software. It has this capability and it works well.

Oct 30, 2009
5:35 pm
#6 bruser :

lovely peace of sw
just set up my test virt machines with tsm and bacula
and works like charm

Jan 10, 2010
3:58 pm
#7 brownwrap :

OK, I must have missed something. The writeup says, ‘first I downloaded mhvtl-2009-04-09.tgz . Fromwhere?

Trackbacks to this post.
Leave a Comment

Name

Email

Website

Previous Post
« Find your partitions and kernel from the GRUB command line
Next Post
Fedora bloat begone! »
Powered by Wordpress   |   Lunated designed by ZenVerse