
This archive contains several text files. Those files contain information
about the recent scanner tests performed by the Virus Test Center at the
University of Hamburg, Germany.

The files are in plain ASCII text format and can be viewed with any
text editor or browser, or printed by just copying them to a printer.

If you need to contact the author of the tests, here is the necessary
information:

        Name:           Vesselin Vladimirov Bontchev
        Organization:   Virus Test Center, University of Hamburg
        Address:        Vogt-Koelln-Strasse 30, rm. 107 C
                        22527 Hamburg, Germany
        Telephone:      +49-40-54715-224
        Fax:            +49-40-54715-226
        e-mail:         bontchev@fbihh.informatik.uni-hamburg.de


Contents of the archive:

README.1ST      - Latest notes.
CONTENTS.TXT    - This file.
HISTORY.TXT     - History of the different versions of the document.
INTRO.TXT       - Introductory text, explaining what this document is all about.
DISCLAIM.TXT    - Disclaimer: what we are *not* trying to do with this document.
TESTABLE.TXT    - Conditions a scanner must fulfil, in order to be tested.
PROTOCOL.TXT    - Detailed description of our virus testing protocol.
WHY_BAD.TXT     - Some reasons why this test is *not* good enough.
SCANNERS.TXT    - A list of the scanners tested, their producers and versions.
PROBLEMS.TXT    - Problems and bugs noticed in those scanners during the tests.
RESULTS.TXT     - Summarized results of the tests.
GOOD_BAD.TXT    - Which scanners are the best and which are the worst.
EPILOGUE.TXT    - Some final words and closing comments.


Version 0.0     - First, very preliminary draft, sent only to the members
                  of CARO (the Computer Anti-virus Researchers' Organisation)
                  for comments.
Version 0.9     - Final beta release, sent only to the members of CARO for
                  review.

Version 1.0     - First public release.
[19-Jul-1994]


Introduction.

Currently, there is an urgent need of professional tests of anti-virus
products. There are several reasons for that. The main one is that the
anti-virus products are not something that the end user is able to
evaluate him/herself. When the user buys a word processor, s/he can
easily see whether it works according to the expectations and whether
it performs the job it is supposed to perform. Not so with the
anti-virus products. An anti-virus product may be installed and
started every day, but its real anti-virus part enters into action
(and shows whether it is any good) only during a real virus attack.
And, regardless of all the media hype, computer viruses are still
relatively rare.  A user could use an anti-virus product a whole year,
if not more, without needing its anti-virus capabilities to stop a
virus attack.

Another reason is that an anti-virus product is extremely difficult to
test. In order to test a word processor, one only needs the manual and
some (potentially big) text files. In order to test an anti-virus
product, one needs a lot of things. First of all, the tester of such a
product must have a deep and intimate knowledge of how computer
viruses work, what are their methods of attack, and what are the
methods to thwart those attacks. The tester must know the principles
on which the anti-virus products work. At last, but not least, the
tester must have access to a fairly rich and well-organized virus
collection. The ideal person who has all of the above is the
anti-virus researcher.

Unfortunately, the anti-virus researchers are hard to come by. Most of
them are busy developing and selling their own products. As such, they
cannot test other people's anti-virus products - because the results
will be always biased towards their own. Therefore, one needs an
independent anti-virus researcher, in order to test an anti-virus
product properly. The number of independent anti-virus researchers in
the world can probably be counted on the fingers of one hand.

Yet another problem is obtaining the necessary resources for a good
anti-virus product test. As mentioned, those tests are very difficult
to perform. The require a lot of disk space, a variety of hardware, a
lot of man-hours to complete. The main question is - how to get the
money to fund all this?

One solution is to have the anti-virus companies pay for the tests.
After all, the results are usually very usful to them (in the form of
bug reports), and sometimes can be used for advertising.
Unfortunately, if the tests are sponsored by an anti-virus company,
the end users tend not to believe them - because they have all the
right to feel that the tests have been biased to show the product of
that company in a better light.  Indeed, why would a company pay money
just to have it demonstrated to the general public that their product
is bad?

Another solution is to have the users of the test results to pay for
the tests - regardless of whether they are an anti-virus company that
just wants to see how well their product performs, compared to others,
or if they are end users, trying to select "the best" anti-virus
product. The main problem with this solution is that, in order to
obtain some sellable results, one need money in advance - to do all
the tests.

We have had the chance to use the facilities of the Virus Test Center
at the University of Hamburg to do the tests. Of course, a lot of
things are still missing (mainly man-power, time, and disk space),
which has resulted in the tests being not as good as we would like
them to be. Nevertheless, we have decided to distribute the results
for free. Of course, if you like them and are in a position to be able
to donate money or hardware to the VTC-Hamburg - we will highly
appreciate this.

One last problem with the anti-virus products, especially those of the
scanner type.  They are modified very often.  This means that their
production cycle is forced to be shorter than for other kinds of
software products.  Usually, the part that is undercut is the quality
control.  If it is too difficult for the end user to assess the
quality of the product, it is often too tempting to put more efforts
into making the product to look pretty, instead of making it a strong
anti-virus tool.  Therefore, it is urgent that professional tests of
anti-virus products are performed, and the results - published, so
that the general public can see what they are really paying for.

Unfortunately, even for the competent anti-virus researcher,
performing a professional test of an anti-virus product is often a too
difficult, nearly impossible task. Such products often consist of
several parts - scanners, monitoring programs, integrity checkers. The
latter two kinds of programs must be tested how well they perform
against each of the known attacks against that particular kind of
anti-virus defense. Just implementing those attacks is a difficult and
tedious job.

Usually the part of the product that is the easiest to test is the
scanner. Even that should be done by a professional anti-virus
researcher, instead of the usual magazine reviewer, because there are
a lot of pitfals to watch for. The full description of how a
professional test of an anti-virus product should be performed is
outside the scope of this document and will be a subject of a
different paper.

Nevertheless, the urgent need for good tests of anti-virus products
prompted us to use our knowledge and technical facilities to test some
of the popular products on the market. This document contains the
results of those tests. Our intent is to update it periodically, as
new anti-virus products, or new versions of the old anti-virus
products appear.

Please, note that the quality of our tests is far from perfect - refer
to the file WHY_BAD.TXT for some points on what is missing from our
tests. Nevertheless, we feel that the results that we can provide are
of supperior quality than many so-called reviews of anti-virus
products that we have seen so far. We are concentrating our efforts on
the anti-virus side of the problem and leave the evaluation of the
pretty user interfaces and the structure of the manuals to the
magazine reviewers.

We hope that our results will help the end user to select a better
product to protect him/her from computer viruses. Whether we have
succeeded to achieve our goal, only the users themselves can tell.


Copyright, License, and Disclaimer.

This publication is (C) Copyright 1994 by Vesselin Bontchev and the
Virus Test Center at the University of Hamburg.  Permission is granted
to everybody to distribute copies of this information in electronic
form, provided that this is done for free, the contents of the
information is not changed in any way, and the source of the
information is clearly mentioned.  Those who want to re-publish the
information as a whole or in parts in printed form, MUST contact
Vesselin Bontchev or the VTC-Hamburg and obtain permission first.

No responsibility is assumed by the publisher for any injury and/or
damage to persons or property as a matter of products liability,
negligence or otherwise, or from any use or operation of any methods,
products, instructions or idea contained in the material herein.


In order to be easily testable, a scanner must conform to the set of
conditions listed below. We believe that those conditions are fairly
reasonable, are not too restrictive, and are useful to both the end
users and the developpers of the product, because they allow them to
test it more easily.

Several of the scanners tested for this release of the test results do
*not* conform to those conditions. This makes the task to test those
scanners very difficult, annoying, and time-consuming. Therefore,
products that do not conform to those conditions will be discontinued
in the next releases of the test results. We label them as "not good
enough even to be worth testing". The users are strongly advised to
avoid those products.

Here is the list of conditions.

1) The scanner must be able to create a report file.

2) In the *beginning* (not at the end!) of this report file, the
scanner must list the directory path that has been scanned.

3) The scanner must be able to list in the report file all objects
that are scanned - not only those that it thinks are infected.

4) The full path of the scanned files must be present in the report
file. Long paths MUST NOT be abbreviated, e.g. by using "..." instead
of several intermediate directory names. Shortening the file paths is
acceptable when displaying them on the screen, but *not* in the
report file.

5) The scanner must be able to scan files with extensions defined by
the user, or if it is not so, it must at least scan the files with
extensions COM, EXE, SYS, and BAT.

6) The scanner must be able to run in "scan-only" mode. If its default
mode is to automatically disinfect all viruses found, there should be
an option to run it in "scan-only" (i.e., no disinfection) mode.

7) The scanner must be able to run unattended - and not to stop on
each infected object and request user input. When the program is ready
with the scan, it must be able to exit automatically and not wait for
additional user intervention.

8) The scanner must be able to run from the command line, scan a
subdirectory tree (not just whole drives) and create a report file
with a name and location supplied by the user.

9) If the scanner issues an audible alarm each time it detects a
virus, there should be a way to turn the sound off. This is not
necessary if the alarm is issued only once - at the end of the
scanning, but the alarm should be able to go away by itself, i.e.
without requiring user intervention.

10) The scanner must be able to run without problems on a huge
directory tree - i.e., something like 7,000 directories containing
20,000 files shouldn't be a problem for it.

11) The only limit of the size of the report file that the scanner
creates must be the amount of free disk space.

12) It should be possible to scan multiple diskettes without leaving
the scanner. The scanner should prompt the user to change the
diskettes. It must request ONE AND THE SAME input from the user
between two diskettes, regardless of whether a virus is found or not.

13) The report file generated when scanning multiple diskettes must
contain information about all the scanned diskettes - not only about
the infected ones, or only about the last one.


Testing Protocol.

Hardware used:

A 50 MHz 486 clone with 8 Mbytes of RAM, one 5.25" 1.2 Mbyte and one
3.5" 1.44 Mbyte floppy disk drives, one 311 Mbyte and one 440 Mbyte
hard disks, running under DR-DOS 6.0 (patched to be able to run
Windows 3.1), QEMM 6.01, and 4DOS 4.02. The virus collection occupies
one 260 Mbyte partition on the second hard disk. The scanners are on a
second, 180 Mbyte partition on the same hard disk.

Testing the scanners on file infecting viruses.

The viruses are stored in a huge subdirectory tree, the hierarchical
structure of which reflects the CARO virus naming scheme, with the
samples of each virus stored in the leaf directories of the tree. A
virus can be (and usually is) represented by more than one replicant,
although the different viruses are not represented by one and the same
number of replicants. All replicants that contain one and the same
virus, are stored in one and the same directory. If two files are in
two different directories, this means that they contain two different
viruses. All efforts have been made to ensure that the samples used
during the test are natural replicants of working viruses - no Germs,
or Corrupted files, or Intended viruses. Nevertheless, it is possible
that we have made some mistakes in this aspect (although we believe
that there are much fewer such mistakes then in other tests we have
seen). If somebody notices any mistakes of this kind, we shall
appreciate being told about them.

Each scanner is run on this directory tree and the resulting report
file is preprocessed. The preprocessing is done with a set of batch
files, some Unix utiltities ported to DOS (sort, join, cut, paste,
awk), and a set of awk scripts. The scripts are available in the
archive SCRIPTS.ZIP. The preprocessed report contains three columns.
The first column contains the directory and the file name of each file
containing a virus. The second column contains the full standard CARO
virus name of the virus contained in the file. The third column
contains information of how the particular scanner report that virus,
or blank if the scanner does not detect a virus in that particular
file.

Not the whole output of the scanner is contained in the third column,
because this output often tends to be too verbose. We have put there
only the distilled information that we have judged important for that
particular scanner. If we have missed some important information, we
shall appreciate being told about it.

Testing the scanners on boot sector infecting viruses.

The boot sector viruses are kept in a similar subdirectory tree, as
files, containing the images of the infected boot sectors. For the
purposes of the test, we used a program, called SimBoot, developped by
Dmitry Gryaznov. This program is still under development and is not
available to the general public, but we will make it available to
those producers of the scanners, who have reasons to suspect that the
program has unfairly interferred with their product and has not
allowed it to be tested properly.

The program takes a file, the first 512 bytes of which are supposed to
contain the first sector of a boot sector virus. It then emulates a
blank, formatted floppy disk in drive A:, the boot sector of which is
replaced by the image in the file. If the file is smaller than 512
bytes, it is padded with zeroes. If the image contains a valid
diskette BPB which indicates a particular diskette size, a diskette
with that particular size is emulated. If a valid BPB is not found, a
360 Kb diskette is emulated. Currently only the first sector of the
boot sector virus is put on the emulated diskette. The program SimBoot
is able to handle complete viruses, consisting of several sectors, but
this requires that the file image of the virus conforms to a
particular format. We did not have the time to prepare all our boot
sector viruses in this way, although we are considering to do this in
the future. One major flaw of this approach is that hard disk, and
respectively MBRs are not emulated. The testing of a virus which
infects only MBRs (e.g., Tequila) but not boot sectors of floppy
disks, is still done by putting an image of the infected MBR on the
boot sector of the simulated diskette. We understand that this is not
very correct - a scanner may refuse to look for a particular virus on
a diskette boot sector, if it knows that this particular virus just
cannot be there. The author of SimBoot is considering to improve it in
the future, in order to make it able to simulate hard disks too.

Once SimBoot creates the simulated infected diskette, it runs the
scanner to be tested, as specified in the configuration file for this
scanner. (The configuration files are available in the archive
SCRIPTS.ZIP.) The scanner is supposed to scan the diskette (SimBoot
intercepts all INT 13h requests to drive A: and redirects them to
access the simulated diskette), report its status in the report file,
and prompt the user to insert the next diskette to be scanned.

SimBoot intercepts the prompt and simulates user input from the
keyboard. Both the prompt and the required user input are specified in
the configuration file for each scanner. SimBoot is able to handle
scanners that write their prompts directly to the video RAM. It is
also able to handle scanners that poll directly the keyboard when
waiting for user input, instead of using the BIOS. Unfortunately, this
capability does not work under QEMM. SimBoot is able even to simulate
changing the status of the floppy drive from Closed to Open and then
again to Closed, in order to handle those scanners which poll the
DiskChanged line, in order to figure out when the user has put a new
diskette.

The resulting report of each scanner is further preprocessed with a
similar set of batch files and awk scripts as the report of the file
virus scanning.

Creating the final summary of the results.

The files containing the preprocessed information mentioned above are
huge. They are not included in this archive. Instead, they were made
available via anonymous ftp from our site. The name of the site is
ftp.informatik.uni-hamburg.de, the IP address is 134.100.4.42, the
name of the directory containing the results is /pub/virus/texts/tests,
and the preprocessed report for each scanner (separately for file and
for boot sector infecting viruses) is storred in an archive, the name
of which suggests the name and the version number of the scanner
tested.

Those results were further proprocessed with several awk scripts, in
order to obtain the data for the summaries listed in the file
RESULTS.TXT.


Why are those tests not good enough?

We would like to emphasize that the tests that we did are NOT what we
would call professional tests of the anti-virus products mentioned.
Here are some reasons why.

1) Many of the products used are integrated systems, including
resident components, integrity checkers, and so on.  ONLY the scanner
part of the product was tested.

2) The resident scanners were not tested - only the on-demand ones
were.

3) The ability of the scanners to detect viruses in memory was not
tested.

4) No special tests were made on some important subclasses of viruses,
like only the polymorphic viruses (on a reasonably big number of
replicants), only the viruses known to be in the wild, and so on.

5) The virus collection used is far from ideal - it does not contain
enough samples of each kind of file infectable by each of the viruses.
We do not mean just COM/EXE/SYS/BAT replicants, but also such things
like very small and very large files, EXE files with internal overlay
structure, and so on.

6) No testing has been done of the disinfection capabilities of
the scanners.

7) No tests have been done to see how well do the scanners perform
when a stealth virus is resident in memory.

8) No attempt has been made to evaluate the user interface of the
scanner, although when it is annoyingly awkward, this is mentioned.
We believe that any normal user or at least a professional reviewer is
able to test this, and have selected to concentrate our efforts in
testing the anti-virus part of the scanners - something that only a
competent anti-virus researcher is able to do.

Therefore, the results obtained have only a limited value.

What are those tests good for, then?

Regardless of the drawbacks mentioned above, we believe that our tests
are of some value.

1) Probably the most valuable part is the naming cross-reference.  It
can help the producers of the scanners to become compliant with the
CARO virus naming scheme and can be used by the users to figure out
which virus they have exactly, after their favorite scanner reports
some name.

2) The tests do provide some overall impression of how good a scanner
is at detecting viruses.  If the results of these tests show that
scanner X has a detection rate of 97.5%, while scanner Y has a
detection rate of 96%, this does not necessarily mean that the latter
is worse than the former.  It just means that the former has shown
slightly better results on this particular virus collection and that
both scanners have a very high detection rate.	However, if the
results show that scanner X is excellent, while scanner Y is total
junk, then those results are pretty reliable.  As Dr.  Alan Solomon
says, to pick a good virus scanner, you don't need to know whether it
detects 96% or 97% of the known viruses - you need to know whether it
is pretty good or very bad.  Most of the existing scanners can be very
easily divided into those two categories.


This is a complete list of the scanners used during the test, with
their version number, producer, and code name, used to abbreviate the
scanner in the table of final results.  If some producer finds that we
have tested a version of their scanner which is too old, they should
feel free to send us an update.

If some producer is upset that we have not tested their product, they
should feel free to send us the product for testing, *if* it fulfils
the set of basic conditions listed in the file TESTABLE.TXT. Please,
note that some of the scanners tested for this release of the test
results do *not* conform to the conditions mentioned above. Since
testing such scanners is too difficult and time-consumming, those
products will be discontinued in the future releases of the test
results, if the respective producers of those scanners do not make
them conform to the said conditions. Besides, as you can see from the
final results, those products usually tend to be buggy, unreliable,
and to have a relatively low detection rate.

Scanner/product name:   VirusHunter
Version number:         915
Code name:              AIDSTEST
Producer:               DialogueScience, Inc., Russia
Status:                 Commercial
Available from:         Producer
Contact:                DialogueScience Inc., Rm. 103a,
                        40 Vavilova ul.
                        117967, GSP-1, Moskow
                        Russia
                        E-mail: lyu@dials.msk.su
Options:                /pfvirs.vht

Scanner/product name:   LGuard / AVAST!
Version number:         6.20
Code name:              AVAST
Producer:               ALWIL Software, Czech Republic
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                /ecom;exe;sys;bat /b /s /p /rfvirs.ast

Scanner/product name:   SUCH / AntiVirenKit 5
Version number:         5.01
Code name:              AVK
Producer:               G DATA, Germany
Status:                 Commercial
Available from:         Producer
Contact:                G DATA Software GmbH
                        Siemensstr. 16
                        D-44793 Bochum
                        Germany
Options:                /nomem /auto_such /eng /no_boot /no_pt

Scanner/product name:   AntiVirus Pro
Version number:         2.00d+
Code name:              AVP
Producer:               KAMI Corp., Russia
Status:                 Shareware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/progs/
                        avp_200.zip, avp_200c.zip, avp_200d.zip, pm940605, smeg.zip, dr_et.zip
Contact:                Tel.: +7 (095)278-9949
                        Fax:  +7 (095)278-9418
                        E-mail: eugene@kamis.msk.su
Options:                /t /m /p /b /q /s /y /w=fvirs.ksp

Scanner/product name:   AVScan
Version number:         1.62
Code name:              AVSCAN
Producer:               H+BEDV Datentechnik GmbH, Germany
Status:                 Freeware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/prog/avscn158.zip
Contact:                D-88069 Tettnang
                        Germany
                        Tel.: +49-07542-93040
                        Fax:  +49-07542-52510
Options:                /s /u /nm /nc /nb /nbr /q /r /ex"com exe sys bat boo" /lfvirs.avs

Scanner/product name:   BRM_Scan
Version number:         3.13p
Code name:              BRM_SCAN
Producer:               BRM Technologies, Israel
Status:                 under development
Available from:         n/a
Contact:                E-mail: yuval@brm.co.il
Options:                -e -f -m -x=bat -s=fvirs.bsc

Scanner/product name:   ChkBoot / FixUtilities
Version number:         2.1c / 6
Code name:              CHKBOOT
Producer:               Padgett Peterson
Status:                 Freeware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/progs/fixutil6.zip
Contact:
Options:                none

Scanner/product name:   Central Point Anti-Virus
Version number:         2.1
Code name:              CPAV
Producer:               Central Point Software / Symantec
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                n/a

Scanner/product name:   Cure / AntiVirus Plus
Version number:         4.20.15
Code name:              CURE
Producer:               IRIS Software
Status:                 Commercial
Available from:         Producer
Contact:                6 Hamavo st.
                        Givataim 53303
                        Israel
                        Tel.: +972-3-5715319
                        Fax:  +972-3-318731
Options:                /f /o /rep /q /n

Scanner/product name:   F-Prot
Version number:         2.12c
Code name:              F-PROT
Producer:               FRISK Software International
Status:                 Freeware/Shareware
Available from:         oak.oakland.edu:/SimTel/msdos/virus/fp-212c.zip
Contact:                Frisk Software International
                        Postholf 7180
                        IS-127 Reykjavik
                        Iceland
                        Fax: +354-1-617274
                        E-mail: frisk@complex.is
Options:                /nomem /list /noboot /nowrap /old /ext=com.exe.sys.boo.bat /report=fvirs.fpr

Scanner/product name:   FindVirus / Dr. Solomon's Anti-Virus ToolKit
Version number:         6.56 with drivers of 14-Jun-94 / ?
Code name:              FINDVIRUS
Producer:               S&S International PLC, UK
Status:                 Commercial
Available from:         Producer
Contact:                Alton House
                        Gatehouse Way
                        Aylesbury
                        Bucks HP19 3XU, UK
                        Tel.: +44-296-318700
                        Fax:  +44-296-318777
                        E-mail: drsolly@sands.co.uk
Options:                /guru /loud /vid /!doboots /!dobats /nm /nc /noboot /yesbreak /oneonly /report=fvirs.sol

Scanner/product name:   Integrity Master
Version number:         2.21b
Code name:              I-MASTER
Producer:               Stiller Research
Status:                 Shareware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/progs/i-m221b.zip
Contact:                2625 Ridgeway St.
                        Tallahassee, FL. 32310-5169
                        U.S.A.
                        CIS: 72571,3352
                        Genie: W.Stiller
                        Prodigy: PHSH44A
Options:                /vm /b /ne /nob /1 /rf=fvirs.ims

Scanner/product name:   IBM Antivirus/DOS
Version number:         1.06
Code name:              IBMAV/DOS
Producer:               IBM
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                -programs -vlog -nb -copenerr -cerr -nrep -nwipe -nfscan -nlmsg -logfvirs.ibm

Scanner/product name:   Microsoft Anti-Virus
Version number:         ?
Code name:              MSAV
Producer:               Microsoft / Central Point / Symantec
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                n/a

Scanner/product name:   Norton Anti-Virus
Version number:         3.0 with updates of March '94
Code name:              NAV
Producer:               Symantec
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                n/a

Scanner/product name:   Norman Virus Control / Norman Data Defense
Version number:         3.41
Code name:              NORMAN
Producer:               Norman Data
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                /b /y /bs- /s /u0 /la /o /n /ebat /lffvirs.nor

Scanner/product name:   PC Vaccine Professional
Version number:         2.02
Code name:              PCVP
Producer:               Computer Security Engineers, Ltd.
Status:                 Commercial
Available from:         Producer
Contact:                P.O. Box 85502
                        NL-2508CE Den Haag
                        The Netherlands
                        Tel.: +31-70-3622269
                        Fax:  +31-70-3652286
Options:                /n /v

Scanner/product name:   VIRUSCAN
Version number:         116
Code name:              SCAN
Producer:               McAfee, Inc.
Status:                 Shareware
Available from:         oak.oakland.edu:/SimTel/msdos/virus/scanv116.zip
Contact:                2710 Walsh Avenue, Suite 200
                        Santa Clara, CA  95051-0963
                        U.S.A.
                        Tel.: +1-408-988-3832
                        Fax:  +1-408-970-9727
                        E-mail: mcafee@mcafee.com
Options:                /nomem /nopause /sub /report fvirs.scn

Scanner/product name:   VIRUSCAN 2.0
Version number:         2.10
Code name:              SCAN2
Producer:               McAfee, Inc.
Status:                 Shareware
Available from:         oak.oakland.edu:/SimTel/msdos/virus/scn-202.zip
Contact:                2710 Walsh Avenue, Suite 200
                        Santa Clara, CA  95051-0963
                        U.S.A.
                        Tel.: +1-408-988-3832
                        Fax:  +1-408-970-9727
                        E-mail: mcafee@mcafee.com
Options:                /nomem /sub /std /append /rpterr /rptcor /report fvirs.sc2

Scanner/product name:   Sweep
Version number:         2.10
Code name:              SWEEP
Producer:               Sophos, UK
Status:                 Commercial
Available from:         Producer
Contact:                Sophos Plc, Oxford, England
                        Tel.: +44-235-559933
                        Fax:  +44-235-559935
Options:                -nb -ndi -nk -p=fvirs.swp

Scanner/product name:   TbScan / TBAV
Version number:         6.22
Code name:              TBSCAN
Producer:               ESaSS B.V., The Netherlands
Status:                 Shareware
Available from:         oak.oakland.edu:/SimTel/msdos/virus/tbav620.zip
Contact:                P.O. Box 1380
                        6501 BJ Nijmegen
                        The Netherlands
                        Tel.: +31-80-787881
                        Fax:  +31-80-789186
                        E-mail: veldman@esass.iaf.nl
                        CIS:    100140,3046
                        FidoNet: 2:280/200
Options:                in nb nm nh ba el na ol lo ll=4 ln=fvirs.tbs

Scanner/product name:   UTScan / Untouchable?
Version number:         32.01
Code name:              UTSCAN
Producer:               BRM Technologies / Fifth Generation Systems / Symantec
Status:                 Commercial
Available from:         Producer
Contact:                E-mail: yuval@brm.co.il
Options:                -b -sa- -vr -$ -r:fvirs.uts

Scanner/product name:   V-Care
Version number:         4.41
Code name:              V-CARE
Producer:               NSE Software, Israel
Status:                 Commercial
Available from:         Producer
Contact:                Tel.: +972-3-575-6324
Options:                none

Scanner/product name:   VirusBuster Lite
Version number:         1.60
Code name:              VB-LITE
Producer:               Leprechaun Software Pty Ltd, Australia
Status:                 Shareware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/progs/vb-lite.zip
Contact:                PO Box 826
                        Capalaba
                        4157
                        Australia
                        Tel.: +61-7-823-1300
                        Fax:  +61-7-823-1233
Options:                /c+ /f+ /l+ /o+ /r+

Scanner/product name:   VFSLite / Virus Detection System
Version number:         3.0m
Code name:              VDS
Producer:               VDS Advanced Research Group
Status:                 Shareware
Available from:         oak.oakland.edu:/SimTel/msdos/virus/vds30m.zip
Contact:                P.O. Box 9393
                        Baltimore
                        MD 21228
                        USA
                        Fax: +1-717-846-2533
Options:                -q -n -rfvirs.vds -zfvirs.vds

Scanner/product name:   VET
Version number:         7.632
Code name:              VET
Producer:               Cybec Pty Ltd, Australia
Status:                 Commercial
Available from:         Producer
Contact:                P.O. Box 205
                        Hampton
                        VIC 3188
                        Australia
                        Tel.: 521-0655
                        Fax:  521-0727
Options:                /e /f /n /r /x /_ /j /h=0 /l=fvirs.vet

Scanner/product name:   VirC
Version number:         4.06
Code name:              VIRC
Producer:               National Laboratory of Computer Virology, Bulgaria
Status:                 Commercial
Available from:         Producer
Contact:                Acad. G. Bontchev St. bl. 8 rm. 104
                        Sofia
                        Bulgaria
                        Tel.: +359-2-719212
Options:                /nomem /list=fvirs.vrc

Scanner/product name:   VDScan / ViruScope
Version number:         4.33
Code name:              VIRUSCOPE
Producer:               PULS Development, Austria
Status:                 Commercial
Available from:         Producer
Contact:                Ing. Leopold Puchas
                        Hessegasse 30-32/27
                        A-1220 Wien
                        Austria
Options:                /sn /m- /q

Scanner/product name:   Buster / VirusBuster
Version number:         4.03.02
Code name:              VIRUSBUST
Producer:               Leprechaun Software Pty Ltd, Australia
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                /pd:\filevirs /b /h /m- /x /w

Scanner/product name:   VIS
Version number:         4.3
Code name:              VIS
Producer:               Total Control, UK
Status:                 Commercial
Available from:         Producer
Contact:                ?
Options:                /b /p /q /r

Scanner/product name:   VPCScan / Virex-PC
Version number:         2.93
Code name:              VPCSCAN
Producer:               Datawatch Corp.
Status:                 Shareware
Available from:         ftp.informatik.uni-hamburg.de:/pub/virus/progs/ ?
Contact:                Triangle Software Division
                        P.O. Box 13984
                        Research Triangle Park,
                        NC  27709-3984
                        USA
                        Tel.: +1-919-5490711
                        Fax:  +1-919-5490065
Options:                -f -m -t -s -bi -!n -!c -!j -!b -rfvirs.vrx

Scanner/product name:   XScan
Version number:         2.12
Code name:              XSCAN
Producer:               ANYWARE SOFTWARE CO.
Status:                 Commercial
Available from:         Producer
Contact:                Tel.: +34-1-556-9215
                        Fax:  +34-1-556-1404
Options:                /report fvirs.xsc


The following problems have been encountered with the different
scanners during the tests.


AIDSTEST:
---------

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

4) The scanner crashes when scanning some boot sectors with incorrect
BIOS Parameter Block in them, merely two samples of Form. Those are
the same boot sectors that crash SCAN.

All this makes it very difficult and time consuming to test it.
Unless the producer fixes the above problems, we will not test this
scanner any more in the future.


AVAST:
------

1) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3). This makes it very difficult and
time-consuming to test it. If the producer does not fix this problem,
we will probably discontinue testing the product.


AVK:
----

1) The scanner (SUCH) is unable to scan multiple floppies without
exiting (Condition 12).  The interactive scanner (ANTIVIR) polls the
keyboard directly, so it is not possible to use SimBoot's keyboard
simulator to tell it to scan another floppy from the menu.  In order
to get at least some boot sector virus test data, we instructed
SimBoot to start the scanner anew for each boot sector image.  Even
then, the scanner crashed on several boot sector images - exactly
those on which IBMAV/DOS has problems.


AVP:
----

1) The scanner is unable to scan multiple floppies without exiting
(Condition 12). It's keyboard handling routine polls the keyboard
directly, so it is not possible to use SimBoot's keyboard simulator to
tell it to scan another floppy from the menu.

2) When started, the scanner complains that the database CA.-VB is
out-of-date. This database does not contain virus descriptions, so
updating it every three months shouldn't be required.

3) The program -VPRO.EXE crashes when run under DesqView. The main
scanner, -V.EXE runs OK, though.

4) The program is extremely sensitive to the amount of memory
available to it. If the memory is not sufficient, the program becomes
unreliable, like not being able to detect all viruses it normally
does, being unable to disinfect some viruses and so on.

5) When started from a write-protected floppy, the scanner complains
that the floppy is write-protected, even if it is instructed not to
create a report file, and asks for a path to a drive with at least 2
Mb free disk space. This is unacceptable, because starting the scanner
from a write-protected floppy is the most secure and recommended way
to run it.


AVSCAN:
-------

None.


BRM_SCAN:
---------

1) Actually, this is not a scanner. It is only a scanning engine which
is under development, with a crude user interface, allowing it to be
tested. Since the user interface was added only for testing purposes
and the will be completely different in the final product, a normal
user certainly wouldn't like it. However, it allowed us very easily to
test it, because it was designed especially to be easily testable and
conformed to all our conditions.

In fact, since this is not a ready product, available on the market,
we weren't sure whether to include it in this report. However, since
we did test it anyway (in order to help the producer in its
development), we had complete test results. They were so impressive,
that we couldn't resist and decided to include it, after verifying
that the producer does not object to it. However, this scanner was not
included in the final classation of "best" and "worst" products.

2) When scanning multiple simulated floppies, the scanner stopped
after the first one, with a message that it cannot read the boot
sector. However, it was able to run perfectly in a DOS window under
DesqView. We do not know what the problem exactly is, but the producer
is strongly suggested to fix it.

3) The scanner crashed when scanning one particular file. The reason
was that this file had an invalid date of last modification (the day
was 0). According to the producers, this is caused by a bug in the
library that comes with the compiler they are using and will be fixed
in a next release.


CHKBOOT:
--------

1) ChkBoot is not a general-purpose scanner; it is a heuristic
analyser for boot sector viruses *only*. Therefore, only boot sector
virus data is available.

2) Since the program is able to output basically only two kinds of
reports about a boot sector (infected/clean), we didn't include it in
the final classation for "best" and "worst" scanner - otherwise it
would have shown best results in reliability of the detection and
worst result in number of different reports.

3) The detection rate was amazing - it missed only 3 viruses. We are
aware that a newer version exists, which does not miss even them, but
Padgett has not released it yet.


CPAV:
-----

1) When started from the command line and instructed to scan our virus
collection, CPAV simply hangs the machine. When started interactively
from the menu and instructed to switch to the drive containing our
virus collection, the program begins to count the existing
directories. When it is almost at the end, it displays a message that
there is not enough memory and hangs the machine. It is obvious that
the scanner is unable to handle big directory structures (Condition
10).

2) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

3) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

4) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

5) The scanner is unable to scan multiple floppies without exiting
(Condition 12). One alternative solution to this problem is to
repeatedly select the floppy drive from the menu. Unfortunately, the
scanner polls directly the keyboard for user input, and SimBoot is
unable to simulate keyboard input in this case.

6) When tested on some polymorphic viruses from our collection, the
scanner simply crashed while examining them.

All the above problems make the product simply *untestable*. We didn't
succeed to obtain any test data for it, or even to run it for that
matter. Having in mind the many other deficiencies that exist in
CPAV, we strongly recommend that people do *not* rely on this product
for virus protection.


CURE:
-----

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not scan SYS and BAT files (Condition 5) and
allows the user to specify only one additional extension to scan.

3) The scanner is unable to scan multiple floppies without exiting
(Condition 12). We tried to start the scanner anew for each of the
simulated floppies, but then it turned out that it does not report
anything in the report file, if it does not consider the floppy as
infected (Condition 13). Therefore, no boot sector virus test data is
available.

4) It is not possible to instruct the scanner from the command line
about the name of the file where the report must be saved (Condition
8).

5) When we started the scanner on our virus collection, it scanned a
few files and aborted with a "Stack overflow" error.

All those problems make the scanner untestable. We will not bother to
test it in the future, unless the producer seriously improves the
product and fixes the problems mentioned above. Meanwhile, we strongly
advise the users against relying on this scanner.


FINDVIRUS:
----------

1) This scanner is the best when it comes to exact virus
identification. Unfortunately, the latest version has a few
identification problems - sometimes it says "identified" with one and
the same name about two different viruses, or with different names
about different samples of one and the same virus.

2) During the boot sector test, the scanner emits a horrible beep each
time it finds a virus on the simulated floppy. There should be a way
to turn the beep off (Condition 9).

3) The program becomes extremely slow when told to identify exactly
some viruses. For instance, it took 12 minutes on a '486-based
computer (!) to scan a *single* file containing a variant of the
Dir_II virus.


F-PROT:
-------

1) The scanning of boot sector viruses often results in "unknown"
variants. It seems that the exact identification for those boot sector
viruses has to be improved.


I-MASTER:
---------

1) The report file generated by the program is huge and extremely
verbose - even when the /1 option is used. It might be difficult for
the user to orient him/herself in it. Automatic preprocessing of the
information *is* possible, although not easy. As an example of how
verbose the report file is - the full report file of the product when
performing virus scan on our collection of boot sector viruses
occupies almost 3 megabytes. The same report preprocessed, so that
only the essential information is extracted from it, occupies about 35
kilobytes.


IBMAV/DOS:
----------

1) When scanning some floppies with boot sector images on them that do
not have a correct BIOS Parameter Block, the scanner causes a DOS
critical error (Abort, Retry, Ignore, Fail?) and regardless of the
user input always aborts. This causes serious problems during the
tests and we urge the producers to fix the problem. Fact is, that
other scanners do not have it. There is a total of 14 boot sectors
causing this problem. They will be provided to the producer on
request. Interestingly, the previous version of the scanner, 1.05,
does not have any problems with those boot sectors.

2) The scanner performs exact identification of a very limited number
of viruses.  However, in several cases I've found this identification
not to be correct - it reports samples of one and the same virus as
different ones.  One particularly frappant case is the Frodo virus -
the program seems to include part of the virus that gets trashed in
its virus maps.  In practice, this means that it won't be able to
disinfect this virus in those cases.


MSAV:
-----

1) When started from the command line and instructed to scan our virus
collection, MSAV simply hangs the machine.  When started interactively
from the menu and instructed to switch to the drive containing our
virus collection, the program begins to count the existing
directories.  When it is about in the middle, it displays a message
that there is not enough memory and stops.  If an atempt is made to
scan the drive at that time, the program does not scan anything and
displays its (empty) Statistics window.  Unline CPAV, though, MSAV
does not hang in this case. Nevertheless it is obvious that the
scanner is unable to handle big directory structures (Condition 10).

2) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

3) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

4) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

5) The scanner is unable to scan multiple floppies without exiting
(Condition 12). One alternative solution to this problem is to
repeatedly select the floppy drive from the menu. Unfortunately, the
scanner polls directly the keyboard for user input, and SimBoot is
unable to simulate keyboard input in this case.

6) When tested on some polymorphic viruses from our collection, the
scanner simply crashed while examining them.

All the above problems make the product simply *untestable*. We didn't
succeed to obtain any test data for it, or even to run it for that
matter. Having in mind the many other deficiencies that exist in
MSAV, we strongly recommend that people do *not* rely on this product
for virus protection.


NAV:
----

The program suffers from a brain-damaged design, especially in the
part that is responsible for the reports, which makes it completely
unsuitable for testing, and often even for ordinary use.

1) It is impossible to run the program from the command line and tell
it to scan a particular directory subtree and to create a report file
with a name supplied by the user, which lists all files that have been
scanned (Condition 8).  There is a /report option, but it is bogus -
it simply doesn't work from the command line.  OK, we made an
exception and ran the program interactively.  At this point the users
who plan to use the product from batch files, unattended, should
already look into something else.

2) It is not possible to tell the program to list in the report file
ALL files that have been scanned (Condition 3).  It lists only the
files that it thinks contain a virus.  We were given the idea to run
the program in "inoculate" mode, and when it finishes, refuse the
inoculation.  This creates a report file, where the name of each file
is listed, the name of the virus found in it (if any), and its
inoculation status (in this case - not inoculated).  BTW, Symantec
uses the term "inoculation" to mean that a checksum is computed for
the file and stored in a database.  This is clearly misleading; in the
anti-virus industry the term is usually used to mean that something is
attached physically to the protected files (i.e., they are modified),
in order to make them virus-resistent.

3) OK, we tried this too.  In this mode, the report for each file
occupies multiple lines (file name, virus name, inoculation status),
which sometimes makes the automatic preprocessing of the report a bit
problematic.  This is actually not a problem, however, if those
multiple lines conform to some reasonable rules - like each line is
contains a tag that identifies it, at least the line with the file
name is always present, and so on.  At a first look, it seems that the
report generated by NAV 3.0 when used in this mode conforms to the
above requirements.  But only at a first look!

Carefull examination showed that the report is severly screwed up.
First of all, the line that is supposed to contain the file name is
sometimes split in two. That is, sometimes you have

        Name: c:\some\dir\with\a\file

and sometimes

        Name:
        c:\another\dir\longer\this\time\with\another\file

4) OK, we figured out how to handle this too. But this is not all!
Sometimes, when the full file path is rather long, it is simply
truncated! In extreme cases this can mean that the file name is simply
missing from the report file and only part of the directory path is
there. So much for those who expect to get a reliable report of all
their infected files. We can handle weird report file formats, but we
certainly cannot handle information that is missing or corrupted. The
final kludge that allowed us to create some kind of report file with
all the information in it, was to assign a drive to the directory
subtree that we wanted to scan, and then tell NAV to scan that
"drive". Since the full paths of the files on this "drive" were
shorter, they were not truncated. However, in real-life situations,
it is clearly unacceptable to expect such kludges from the user.

5) The scanner is unable to scan multiple floppies in sequence
(Condition 12).  We couldn't do it interactively from the menu either,
because it polls directly the keyboard for user input and therefore
SimBoot's keyboard simulator is useless in this case.

6) The scanner has a few other drawbacks (for instance, it is unable
to run on a 8088, so if you have XTs in your organisation and intend
to use a single product for virus protection - forget about this one).

7) Some time ago it was claimed that, unlike NAV 2.1, version 3.0 of
the product is able to detect all MtE-based viruses reliably. Our
tests demonstrate this claim to be wrong - the virus Destructor:MtE is
not detected at all, and the viruses Ludwig.[A-C]:MtE are not detected
reliably.

All these problems make testing the scanner extremely difficult and
time-consuming. Having in mind the relatively low detection rate that
the scanner demonstrated, we decided not to bother to test it in the
future, unless Symantec fixes all the problems reported above. We
strongly discourage the users from relying on this scanner for virus
protection.


NORMAN:
-------

1) When we attempted to run this scanner, it greeted us with the
message "This DEMO-version has expired".  (We obtained a copy of the
scanner from the producers on the Hannover Computer Fair.) Since we do
not review crippleware, we decided not to bother to set the date on
the computer back.  Previous tests showed that the detection rate is
nothing particularly high - something about 75%.


PCVP:
-----

1) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

2) The scanner does not allow the user to specify the name of the
report file (Condition 8).  Instead, it always writes the report to
the file SCAN_x in the directory from which PCVP has been started, 'x'
being the drive on which the directory subtree being scanned resides.

3) The scanner has bugs when reporting viruses written in a high-level
language, that spread in compressed form (e.g., LZEXE-compressed).
Versions 1.x of the scanner had the problem of not reporting such
files as infected at all in the report file. This bug has been
reported to the producer, and they promised to have it fixed. The
"fix" in version 2.01 has another problem - now those files are
sometimes reported once, and sometimes reported twice - once as not
infected and then as infected. Obviously, the producer has never
bothered to test whether the fix actually works.

4) It is interesting to note that the latest version of the scanner
that we tested (2.02) demonstrated almost 10% *lower* detection than
the previous version (2.01), when used on the same virus collection.

The producer is strongly suggested to fix the above problems, or we
will discontinue testing the product, as the test being too
time-consuming and not worth the effort.


SCAN:
-----

1) The scanner is unable to scan BAT files and does not allow the user
to indicate which extensions are to be scanned (Condition 5).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner crashes when scanning some boot sectors with incorrect
BIOS Parameter Block in them, merely two samples of Form. The error
that occurs is "Divide error". However, since there were only two such
samples, we decided to remove them and run the test nevertheless. The
two samples crashing the scanner are clearly marked as such in the
preprocessed report file and will be made available to the producer,
on their request.

The above problems make the scanner very difficult and time-consuming
to test. Having in mind that its detection has dropped under the "good
enough" limit, we think that it is not worth the effort to test this
scanner any more, unless the producer takes care to fix the above
problems.


SCAN2:
------

1) The scanner is unable to scan BAT files and does not allow the user
to indicate which extensions are to be scanned (Condition 5).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

4) Each time the scanner detects a virus on the simulated floppy, it
emits a horrible sound. There is no way to turn this sound off
(Condition 9).

5) Curiously, version 2.10 of the scanner demonstrated results that
were significantly worse than those demonstrated by version 2.02. It
seems that the program is getting worse and worse with time.

All those problems make it very time-consumming to test the scanner.
Since it also demonstrated a relatively low detection rate and
*extremely* low reliability of the detection, we decided not to bother
to test it in the future, unless the producer seriously improves it,
and fixes the problems mentioned above. We discourage the users from
relying on this product for virus protection - it leaves the
impression to be unfinished.


SWEEP:
------

1) The scanner is unable to scan BAT files and does not allow the user
to indicate which extensions are to be scanned (Condition 5).

2) The scanner is able to scan multiple floppies without exiting
(Condition 12).  However, SimBoot was unable to intercept the prompt
between each two floppies and/or to simulate keyboard input.  We had
to "press the any key" manually. This was very annoying thing to do
645 times and unless the problem gets fixed, we do not intend to do it
any more in the future.

3) The scanner is unable to scan a subdirectory tree - it can scan
only whole drives (Condition 8).

4) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

All those problems make it very time- and effort-consuming to test the
scanner. We have decided not to test it any more in the future, unless
the producer fixes the problems mentioned above.


TBSCAN:
-------

1) The scanner crashes when scanning some boot sectors with incorrect
BIOS Parameter Block in them.  One of the samples crashes the scanner
immediately, while another one causes it to crash on exist.  In both
cases, no report file is available after the crash.  The two samples
that crash the scanner are *not* the same two that crash SCAN.  This
problem has been reported to the producer when the scanner was at
version 6.02, yet the problem is still not fixed.

2) We would have removed the two samples, as we did with the test of
SCAN, but there was another problem.  Several samples cause the
scanner to flash an "Access denied" error message and not to report
anything in the reprot file for that simulated diskette.  Since there
were many such cases, and since there is no trace of them in the
report file, determining which exactly are the problematic samples
would be too difficult.  Therefore, no boot test data for this scanner
is available.


UTSCAN:
-------

1) The scanner is unable to scan multiple floppies without exiting
(Condition 12).  It also polls the keyboard directly when expecting
user input, so it is not possible to use SimBoot's keyboard simulator
to tell it to scan another floppy from the menu. We tried to start the
scanner anew for each of the simulated floppies, but then there were
other problems. First, the scanner sometimes "fooled itself", by
reporting a virus in memory after scanning an infected boot sector.
Second, it crashed when scanning some floppies with incorrect BPB. At
this point we decided that it is not worth the effort to try to test
the scanner for boot sector virus detection.  Therefore, no boot
sector tests have been done and no such results are available.

2) The computer containing the virus collection runs a disk encryption
driver, which reserves 5 DOS logical drives for mounting encrypted
volumes. Those drives are reported as "removable" by DOS (i.e., they
behave like floppy disk drives). If an attempt is made to access them
when no encrypted volumes are mounted, the driver returns a "Bad drive
unit" error. This confuses UTScan and it offers no other opportunity
than to exit without scanning anything. In order to perform the tests,
we had to remove the encryption driver.


V-CARE:
-------

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

4) It is not possible to indicate from the command line where the
scanner should put the report file (Condition 8).  The scanner always
creates a report file with a fixed name, in the root directory of the
drive being scanned.  We wonder how do the users of the product create
report files when scanning write-protected floppies... Anyway, this
prevented us from testing it on simulated floppies, thus no boot
sector virus test data is available for this scanner.

The above problems make the scanner difficult and time-consuming to
test. Having in mind the very low detection rate that it showed, we
decided not to test this scanner any more in the future, unless the
producer seriously improves it.


VB-LITE:
--------

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

3) The scanner does not list the full paths of the files it reports in
the report file - the long ones are shortened by replacing some part
in the middle with "...". This makes it impossible to correctly
process the report, because some essential information is missing from
it. Therefore, we can provide information only about the number of
detected samples, not about the number of detected viruses.

4) The scanner is able to scan multiple floppies without exiting
(Condition 12), but it polls the DriveChanged line to detect diskette
change and also polls the keyboard directly. SimBoot is able to
simulate the DriveChanged line, but keyboard simulation does not work
under QEMM, when the keyboard is accessed directly by the application.
We couldn't run the scanner anew for each of the simulated floppies
either, because each time before exiting, the scanner pauses for about
ten seconds and wait for the user to press a key. Therefore, no boot
sector virus test data is available for this scanner.

The above problems, and especially the third one, make the scanner
almost impossible to test properly. We decided not to test it any more
in the future, unless the producer fixes all of the problems mentioned
above.


VDS:
----

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

3) There seems is an implementation bug in VDS, which causes the contents
of the configuration file to take precedence over the options specified
from the command line. The producer suggested us to use the program
VFSLite from the VDS package, which is only a scanner and does not
have this problem.

4) When instructed to scan multiple floppy disks for viruses, VDS
attempts to remove every virus found. It also displays different
prompts between each two floppies, depending on whether the floppy
that has just been scanned has been detected as infected or not. At
last, when an infected boot sector is found, the report file does not
contain any information about it. All this makes the scanner
untestable with SimBoot. Therefore, no boot sector virus data is
available.


VET:
----

1) The name of one of the files in our collection contains a 8-bit
character ("O-umlaut"). VET claims that it has an illegal name. Seems
that the producer has not heard of non-English versions of DOS. :-)

2) By default the program removes any viruses it detects - something
which can be very annoying to the tester, although it is probably
convenient to the normal user. Fortunately, there is an option to turn
this behaviour off (Condition 6).

3) The scanner is unable to scan BAT files and does not allow the user
to indicate which extensions are to be scanned (Condition 5).


VIRC:
-----

1) When the program is ready with the scanning, it stops and waits for
the user to press a key. This causes problems to let it run unattended
from batch files (Condition 7).

2) When scanning some boot sectors with incorrect BPBs, the program
crashes. Therefore, no boot sector virus test data is available.


VIRUSCOPE:
----------

1) The scanner does not scan BAT files. It seems to allow the user to
indicate which extensions are to be scanned, but we couldn't make this
feature work (Condition 5).

2) It is not possible to indicate from the command line where the
scanner should put the report file. One must create a special config
file, in order to do this (Condition 8).

3) The scanner does not list in the report file the names of all files
that have been scanned - it lists only those it considers to be
infected (Condition 3).

4) The scanner does not list the full path of the directory being
scanned in the beginning of the report file (Condition 2).

5) The scanner is unable to scan multiple floppies in sequence
(Condition 12). We tried starting the scanner anew for each simulated
floppy, but then it crashed very severly on some boot sectors. Having
in mind all the other problems that the scanner demonstrated, we
desided that it is not worth the effort to continue with our attempts
to test it. Therefore, no boot sector virus data is available for this
scanner.

6) Each time the scanner detects a virus on the simulated floppy, it
emits a horrible sound. There is no way to turn this sound off
(Condition 9).

All those problems make it very difficult and time-consuming to test
the scanner. We will discontinue testing it in the future, unless the
producer fixes the above problems.


VIRUSBUST:
----------

1) The scanner is able to scan multiple floppies without exiting
(Condition 12), but it polls the DriveChanged line to detect diskette
change and also polls the keyboard directly. SimBoot is able to
simulate the DriveChanged line, but keyboard simulation does not work
under QEMM, when the keyboard is accessed directly by the application.

2) It is not possible to instruct the scanner from the command line
about the name of the file where the report must be saved (Condition
8).


VIS:
----

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

4) The scanner can scan only whole drives; not a specified directory
tree.

5) The scanner is able to scan multiple floppies in a sequence.
However, the only way to generate a report file is to redirect the
standard output of the scanner to a file. Unfortunately, this means
that the prompt to "press any key" between each two floppies also goes
in this file and is not displayed on the screen. Therefore, SimBoot is
unable to intercept it and simulate a floppy disk change. Thus, we
couldn't produce any boot sector virus test data.

All the above problems make the scanner very difficult and
time-consumming to test. Unless the producer fixes them, we will not
bother to test this scanner any more.


VPCSCAN:
--------

1) The scanner is unable to scan BAT files and does not allow the user
to indicate which extensions are to be scanned (Condition 5).


XSCAN:
------

1) The scanner does not list the path being scanned in the beginning
of the report file (Condition 2).

2) The scanner does not list in the report file the names of all files
that have been scanned (Condition 3).

3) The scanner does not scan BAT files and does not allow the user to
specify which extensions have to be scanned (Condition 5).

4) The scanner can scan only whole drives; not a specified directory
tree.

5) When started, the scanner complains that there is not enough
memory, although there is about 630 Kb conventional RAM available.
This does not prevent the program from running, though.

All this makes it very difficult and time consuming to test it.
Unless the producer fixes the above problems, we will not bother to do
any more tests of this scanner.


Detection rate.

The following table answers to the most commonly asked question about
scanners: "How many viruses does that scanner detect?"

Date: 19-Jul-1994

================================================================================
| Scanner   | Number of File | Number of Boot ! Number (%)     | Number of (%) |
| Codename: | Viruses (%):   | Viruses (%):   ! of Inf. Files: | Inf. Boots:   |
|===========+================+================+================+===============|
| Total:    | 4,235 (100 %)  |   358 (100 %)  ! 16,259 (100 %) |  645 (100 %)  |
|===========+================+================+================+===============|
| AIDSTEST  |   975 ( 23 %)  |   185 ( 52 %)  !  4,850 ( 30 %) |  416 ( 65 %)  |
|-----------+----------------+----------------+----------------+---------------|
| AVAST     | 3,613 ( 85 %)  |   326 ( 91 %)  ! 14,774 ( 91 %) |  608 ( 94 %)  |
|-----------+----------------+----------------+----------------+---------------|
| AVK       | 2,554 ( 60 %)  |   277 ( 77 %)  ! 11,075 ( 68 %) |  530 ( 82 %)  |
|-----------+----------------+----------------+----------------+---------------|
| AVP       | 4,143 ( 98 %)  |       n/a      ! 16,084 ( 99 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| AVSCAN    | 3,385 ( 80 %)  |   344 ( 96 %)  ! 14,150 ( 87 %) |  631 ( 98 %)  |
|-----------+----------------+----------------+----------------+---------------|
| BRM_SCAN  | 4,141 ( 98 %)  |   353 ( 99 %)  ! 16,024 ( 99 %) |  640 ( 99 %)  |
|-----------+----------------+----------------+----------------+---------------|
| CHKBOOT   |      n/a       |   355 ( 99 %)  !      n/a       |  642 ( 99 %)  |
|-----------+----------------+----------------+----------------+---------------|
| CPAV      |      n/a       |       n/a      !      n/a       |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| CURE      |      n/a       |       n/a      !      n/a       |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| FINDVIRUS | 4,038 ( 95 %)  |   352 ( 98 %)  ! 15,855 ( 98 %) |  637 ( 99 %)  |
|-----------+----------------+----------------+----------------+---------------|
| F-PROT    | 4,025 ( 95 %)  |   340 ( 95 %)  ! 15,717 ( 97 %) |  627 ( 97 %)  |
|-----------+----------------+----------------+----------------+---------------|
| I-MASTER  | 3,207 ( 76 %)  |   235 ( 66 %)  ! 13,237 ( 81 %) |  476 ( 74 %)  |
|-----------+----------------+----------------+----------------+---------------|
| IBMAV/DOS | 3,599 ( 85 %)  |   340 ( 95 %)  ! 14,548 ( 89 %) |  624 ( 97 %)  |
|-----------+----------------+----------------+----------------+---------------|
| MSAV      |      n/a       |       n/a      !      n/a       |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| NAV       | 2,747 ( 65 %)  |       n/a      ! 12,585 ( 77 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| NORMAN    |      n/a       |       n/a      !      n/a       |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| PCVP      | 2,877 ( 68 %)  |   225 ( 63 %)  ! 12,687 ( 78 %) |  471 ( 73 %)  |
|-----------+----------------+----------------+----------------+---------------|
| SCAN      | 3,350 ( 79 %)  |   302 ( 84 %)  ! 14,253 ( 88 %) |  576 ( 89 %)  |
|-----------+----------------+----------------+----------------+---------------|
| SCAN2     | 3,018 ( 71 %)  |   277 ( 77 %)  ! 12,279 ( 76 %) |  544 ( 84 %)  |
|-----------+----------------+----------------+----------------+---------------|
| SWEEP     | 3,773 ( 89 %)  |   334 ( 93 %)  ! 15,273 ( 94 %) |  618 ( 96 %)  |
|-----------+----------------+----------------+----------------+---------------|
| TBSCAN    | 3,966 ( 94 %)  |       n/a      ! 15,739 ( 97 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| UTSCAN    | 3,808 ( 90 %)  |       n/a      ! 15,018 ( 92 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| V-CARE    |   875 ( 21 %)  |       n/a      !  4,599 ( 28 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VB-LITE   |      n/a       |       n/a      !  9,256 ( 57 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VDS       | 2,088 ( 49 %)  |       n/a      !  9,763 ( 60 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VET       | 3,255 ( 77 %)  |   312 ( 87 %)  ! 13,609 ( 84 %) |  592 ( 92 %)  |
|-----------+----------------+----------------+----------------+---------------|
| VIRC      |   647 ( 15 %)  |       n/a      !  3,210 ( 20 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VIRUSCOPE | 2,661 ( 63 %)  |       n/a      ! 12,100 ( 74 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VIRUSBUST | 2,131 ( 50 %)  |       n/a      !  9,250 ( 57 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VIS       | 2,457 ( 58 %)  |       n/a      ! 10,578 ( 65 %) |       n/a     |
|-----------+----------------+----------------+----------------+---------------|
| VPCSCAN   | 2,906 ( 69 %)  |   254 ( 71 %)  ! 12,680 ( 80 %) |  518 ( 80 %)  |
|-----------+----------------+----------------+----------------+---------------|
| XSCAN     | 2,964 ( 70 %)  |   304 ( 85 %)  ! 12,965 ( 80 %) |  570 ( 88 %)  |
================================================================================

Explanation of the different columns:

1) "Scanner Codename" is the code name of the scanner as listed in the
   file SCANNERS.TXT.

2) "Number of File Viruses (%)" is the number of different file
   infecting *viruses* in the virus collection used during the tests,
   which have been detected by the particular scanner.  Their
   percentage from the full set of viruses in the collection used for
   the tests is given in parenthesis.  We define two viruses as being
   different if they differ in at least one bit in their
   non-modifiable parts.  For the variably encrypted viruses, the
   virus body has to be decrypted before the comparison is to be
   performed.  For the polymorphic viruses, additionally the part of
   the virus which is modified during the replication process has to
   be ignored.

3) "Number of Boot Viruses (%)" is the number of different boot sector
   *viruses* from the collection used for the test that the scanner
   detects. This field is analogous to field 2, only it lists boot
   sector viruses, not file infecting viruses.

4) "Number (%) of Inf.  Files" is the number of *files* infected with
   file-infecting viruses from the test set, which are detected by
   that particular scanner as being infected.  The percentage of those
   files from the full set of files is given in parenthesis.  We often
   have more than one infected file per virus, but not all viruses are
   represented by the same number of files, so this number does not
   give a good impression of the real detection rate of the scanner.
   It is included here only for completeness.  Of course, it still
   *does* provide some information - usually the better a scanner is,
   the more files it will detect as infected.

5) "Number of (%) Inf. Boots" is the number of infected boot sectors
   in the test set that the scanner detects as infected. This field is
   analogous to filed 4, only it lists infected boot sectors, not
   files.

6) We interpret those results in the following way:

     - detection rate above 90% - the scanner is "excellent"
     - detection rate of 80-90% - the scanner is "good enough"
     - detection rate of 70-80% - the scanner is "not good enough"
     - detection rate of 60-70% - the scanner is "rather bad"
     - detection rate of 50-60% - the scanner is "very bad"
     - detection rate below 50% - the scanner is "useless"


Quality of the detection.

The following table provides information about the quality of the detection.

Date: 19-Jul-1994

==============================================================================
| Scanner     | Number of      | Unreliable  | Unreliable      | Multiple    |
| Codename:   | Diff. Reports: | Detections: | Idenifications: | Detections: |
|=============+================+=============+=================+=============|
| Total:      |   4,235+358    |     0+0     |        0+0      |     0+0     |
|=============+================+=============+=================+=============|
| AIDSTEST    |     631+97     |    45+4     |       41+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| AVAST       |   2,305+174    |    23+2     |      242+1      |   365+29    |
|-------------+----------------+-------------+-----------------+-------------|
| AVK         |   1,315+70     |    98+10    |       64+1      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| AVP         |   3,171+n/a    |    16+n/a   |       88+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| AVSCAN      |   1,796+198    |    18+0     |      119+2      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| BRM_SCAN    |   2,569+166    |     8+0     |       80+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| CHKBOOT     |     n/a+1      |   n/a+0     |      n/a+0      |   n/a+0     |
|-------------+----------------+-------------+-----------------+-------------|
| CPAV        |     n/a+n/a    |   n/a+n/a   |      n/a+n/a    |   n/a+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| CURE        |     n/a+n/a    |   n/a+n/a   |      n/a+n/a    |   n/a+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| FINDVIRUS   |   3,641+339    |    19+0     |      115+4      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| F-PROT      |   3,792+253    |    30+1     |       88+4      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| I-MASTER    |   1,164+68     |    32+1     |       54+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| IBMAV/DOS   |   2,270+212    |    30+9     |       61+1      |    48+14    |
|-------------+----------------+-------------+-----------------+-------------|
| MSAV        |     n/a+n/a    |   n/a+n/a   |      n/a+n/a    |   n/a+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| NAV         |   1,674+n/a    |   107+n/a   |      296+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| NORMAN      |     n/a+n/a    |   n/a+n/a   |      n/a+n/a    |   n/a+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| PCVP        |   1,498+95     |    69+1     |       66+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| SCAN        |   1,654+79     |    70+9     |       65+1      |   288+1     |
|-------------+----------------+-------------+-----------------+-------------|
| SCAN2       |   1,803+117    |   200+3     |      165+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| SWEEP       |   2,670+234    |    27+2     |      107+0      |     0+2     |
|-------------+----------------+-------------+-----------------+-------------|
| TBSCAN      |   1,460+n/a    |    57+n/a   |       77+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| UTSCAN      |   2,169+n/a    |    47+n/a   |       44+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| V-CARE      |     246+n/a    |   124+n/a   |       22+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VB-LITE     |     993+n/a    |   n/a+n/a   |      n/a+n/a    |   n/a+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VDS         |     860+n/a    |   115+n/a   |      103+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VET         |   1,488+162    |    37+2     |       65+0      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| VIRC        |     358+n/a    |    30+n/a   |       32+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VIRUSCOPE   |   1,846+n/a    |   133+n/a   |       72+n/a    |   277+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VIRUSBUSTER |   1,033+n/a    |   112+n/a   |       40+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VIS         |   1,407+n/a    |    51+n/a   |       79+n/a    |     0+n/a   |
|-------------+----------------+-------------+-----------------+-------------|
| VPCSCAN     |   1,599+128    |    88+1     |       97+1      |     0+0     |
|-------------+----------------+-------------+-----------------+-------------|
| XSCAN       |   1,359+80     |    53+1     |       38+1      |    10+0     |
==============================================================================

Explanation of the different columns:

1) "Scanner Codename" is the code name of the scanner as listed in the
   file SCANNERS.TXT.

2) "Number of Diff. Reports" is the number of different *names* reported
   by the particular scanner during the test. Compared with the number of
   *viruses* that the scanner detects, this gives, to some extent, an idea
   about how well the scanner distinguishes between the different virus
   variants.

3) "Unreliable Detections" is the number of viruses which the particular
   scanner does not detect reliably. Our definition of unreliable detection
   is that at least one sample of the virus *is* detected and at least one
   sample of the virus is *not* detected. If all samples of the virus are
   detected, then is is counted as reliable detection. If no samples of the
   virus are detected, then the scanner does not detects the virus at all.
   In some sense, the unreliable detections are more dangerous than the
   cases when the scanner misses the virus completely, because an
   unreliable detection lulls the user into a false sense of security.
   Needless to say, *only* the reliable detections are counted as "Number
   of Viruses" in the first table.

4) "Unreliable Identifications" is the number of cases when the particular
   scanner detects the virus reliably, but does not report all replicants
   of the virus with one and the same name. This number provides some
   information about how well the scanner is able to identify the particular
   virus. Viruses which are not identified reliably will almost certainly
   cause problems during disinfection.

5) "Multiple Detections" is the number of cases when the particular
   scanner reports more than one virus in a file but only a single
   virus is present.

6) The results in each column are presented as ffff+bbb, the first
   number refers to the file infectors and the second - to the boot
   sector infectors.


Here are the "winners" and the "losers" of our tests:


The Winners:
------------

1) The scanner with the highest detection rate is:      AVP

2) The scanner with the most reliable detection is:     AVP

3) The scanner reporting the largest number of
   different virus names is:                            F-PROT

4) Scanners with detection rate of 90% or above
   (in decreasing order of the detection rate):

        AVP, FINDVIRUS, F-PROT, TBSCAN, UTSCAN


The Losers:
-----------

1) The scanner with the lowest detection rate is:       VIRC

2) The scanner with the most unreliable detection is:   SCAN2

3) The scanner reporting the smallest number of
   different virus names is:                            V-CARE

4) The scanner reporting (incorrectly) the largest
   number of multiple detections is:                    AVAST


Conclusion.

When reporting the scanners' problems, we have been intentionally
harsh, in order to emphasize the elementary lack of capabilities that
some (many) anti-virus products have.  Judging from past experience,
we are fairly certain that all anti-virus producers whose scanners are
exposed as being inadequate will attack us, our testing methodology,
our understanding of how their product works, and this document.  They
are quite welcome.  We will really appreciate if we are told how to
perform those actions we couldn't find out how to perform (e.g.,
scanning multiple floppies without leaving the scanners, listing all
scanned files in the report file and not only those that the scanner
considers infected, and so on).  All reasonable corrections will be
considered and applied if possible.  However, objections of the kind
"this is not a bug, it is a feature", "our customers will never need
this capability", "you are the tester, figure out how to test our
product; we don't care", and so on, will NOT be accepted.

At last, we would like to express our hope that the users will find
this document useful.

Antiviral Scanner test performed by the 
University of Hamburg Virus Test Center 
dated 7/20/94. This document is (C) 
Copyright 1994 by Vesselin Bontchev and the 
Virus Test Center at the University of 
Hamburg Germany. See RESULTS.TXT to see how 
your Antiviral Scanner Performed.
Antiviral Scanner test performed by the 
University of Hamburg Virus Test Center 
dated 7/20/94. This document is (C) 
Copyright 1994 by Vesselin Bontchev and the 
Virus Test Center at the University of 
Hamburg Germany. See RESULTS.TXT to see how 
your Antiviral Scanner Performed.
