User's Guide - Version 1.2

User's Guide - Version 1.2
Professional Edition




User's Guide
  Version 1.2
User's Guide - Version 1.2
© 2013 PreEmptive Solutions, LLC
All rights reserved.

Document Version 1.2-20131016
www.preemptive.com


TRADEMARKS

PreEmptive Analytics, PreEmptive Analytics for TFS, Dotfuscator, DashO, the PreEmptive Analytics for TFS logo, the PreEmptive Solutions
logo, the Dotfuscator logo, and the DashO logo are trademarks of PreEmptive Solutions, LLC.

.NET™, Team Foundation Server™, and Visual Studio™ are trademarks of Microsoft, Inc.

Java is a trademark of Oracle, Inc.

All other trademarks are property of their respective owners.

THIS PUBLICATION IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO, IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS PUBLICATION COULD CONTAIN TYPOGRAPHIC ERRORS AND/OR TECHNICAL INACCURACIES. UPDATES AND MODIFICATIONS MAY BE
MADE TO THIS DOCUMENT AND/OR SUPPORTING SOFTWARE AT ANY TIME.

PreEmptive Solutions, LLC has intellectual property rights relating to technology embodied in this product. In particular, and without
limitation, these intellectual property rights may include one or more U.S. patents or pending patent applications in the U.S. and/or other
countries.

This product is distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product may be
reproduced in any form by any means without prior written authorization of PreEmptive Solutions, LLC.
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 3

1        Table of Contents

1.      Table of Contents                                                                3-4
2.      Overview                                                                          5
3.      Installing PreEmptive Analytics for TFS                                           6

     3.1. Prerequisites                                                                  6-7
     3.2. Planning a Production Installation                                             7-9
     3.3. Performing a Basic (single-machine) Installation                             10-11
     3.4. Performing an Advanced Installation                                             11

      3.4.1. Installing the Endpoint                                                   11-12
      3.4.2. Installing the Aggregator                                                 12-13

      3.4.3. Installing the Endpoint and Aggregator Together                           13-14
      3.4.4. Securing the Components                                                   14-17
     3.5. Installing the Visual Studio Components                                      17-18
     3.6. Installing the Web Components                                                   18
4.      Configuring PreEmptive Analytics for TFS                                          19
     4.1. Provisioning Team Projects                                                      19

     4.2. Defining Exception Sets and Rules                                            20-21
      4.2.1. Rule Types                                                                21-22
      4.2.2. Possible Surprises                                                        22-23
     4.3. Configuring the New Data Check Interval                                         23
     4.4. Configuring Custom Data Attachment Limits                                    23-24
     4.5. Making URL changes after installation                                           25

      4.5.1. Change Aggregator Service URL                                             25-31
      4.5.2. Change Fault Query Service URL                                               32
5.      Adding Exception Reporting to an Application                                      33
6.      Examining Incidents in Team Explorer                                           34-35
7.      Upgrading PreEmptive Analytics for TFS                                            36
     7.1. Validating the Upgrade                                                          36
8.      Troubleshooting                                                                37-38
     8.1. Troubleshooting Installation                                                    39
     8.2. Troubleshooting the Endpoint                                                 39-42
     8.3. Troubleshooting the Aggregator                                               43-49

9.      Monitoring                                                                        50
10.     Maintenance                                                                       51
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 4

11.   Index                                           52
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 5

Overview

     PreEmptive Analytics for TFS allows you to integrate feedback-driven development processes into your development workflow. By
     analyzing and aggregating exceptions reported by your production applications and automatically creating TFS work items,
     PreEmptive Analytics for TFS gives you access to unprecedented levels of incident data right where you need it – in Visual Studio,
     TFS Web, and Reporting Services.

     Your instrumented applications will automatically send exception report data to your PreEmptive Analytics for TFS installation as
     errors occur during execution. PreEmptive Analytics for TFS will automatically create or update work items in Microsoft Team
     Foundation Server based on rules and thresholds you define. These work items include details about the exception type, message,
     full stack trace, the contents of any inner exceptions, and the list of all assemblies loaded at the time of the exception, as well as
     the number of such exceptions received. With a small amount of additional configuration, you will also be able to see unique
     instance identifiers (such as the serial number of each application instance that experienced the exception) as well as comments
     about the error and contact information, if provided by the end user.


     Components

     PreEmptive Analytics for TFS is made up of five individual components that are distributed via two installers. Those components
     are shown in dark blue in the diagram below.




        Endpoint: Provides a web service that accepts exception messages and stores them for evaluation. This can be exposed to the
        public internet if needed.

        Aggregator: Periodically queries the Endpoint and evaluates new data against rules which are configured by a machine-local
        configuration (admin) utility, and/or by a project-specific (limited) version of the utility installed as part of the Visual Studio
        Components.

        Visual Studio Components: Provides a Visual Studio control for viewing stack traces and incident details, a custom Team
        Explorer pane for accessing the work items created by PreEmptive Analytics for TFS, and a lightweight version of the
        configuration (admin) utility for configuring project-specific settings.

        Web Components: Provides a control for viewing stack traces and incident details in TFS Web Access.

        Reports: Provides reports (for each TFS project) that summarize the work items created by PreEmptive Analytics for TFS for
        that project.

     Please see Installing PreEmptive Analytics for TFS for more information about how and where to install these components.
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 6

Installing PreEmptive Analytics for TFS

      The PreEmptive Analytics Endpoint and Aggregator are both installed by a single installer
      (PreEmptive.Analytics.Pro.Setup.exe), which can operate in either Basic or Advanced mode. Basic mode is intended
      primarily for evaluation purposes as it installs all the components on a single machine. This installer also places the Web
      Components in the same location as the Aggregator. From there, they must be manually installed in TFS Web Access.

      In Advanced mode, you can choose to install the Endpoint separately from the Aggregator, which is the more typical
      installation approach for production use.

      The Visual Studio Components are distributed by a second installer (PreEmptive.Analytics.VisualStudio.exe). The Visual
      Studio Components should be installed on all machines that have Visual Studio and will be used to access team projects
      that have work items generated by PreEmptive Analytics for TFS.



Prerequisites




      Software requirements

      All components

        Windows Vista, 7, 8, Server 2008, Server 2008 R2, Server 2012, or Server 2012 R2
        .NET 4.0 or 4.5 (.NET 4.0 will be installed automatically if neither version is present)


      Endpoint
        SQL Server 2008, 2008 R2, or 2012; or SQL Express 2008, or 2012
        IIS 7.0, 7.5, or 8.0
        Optional: A valid SSL certificate


      Aggregator
        An accessible Team Foundation Server (2010, 2012, or 2013)
        The TFS Object Model (included when installing Visual Studio)
        An installed and operating PreEmptive Analytics for TFS Endpoint


      Visual Studio Components
        An installed and operating PreEmptive Analytics for TFS Endpoint and Aggregator
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 7

         Visual Studio 2010, 2012, or 2013


      Specific user account requirements
      The list below enumerates the access requirements needed for PA for TFS installation and operation. A single account can be
      used to satisfy all requirements.


      Installation requirements
         A Windows user account with rights to install software on the Endpoint and Aggregator machines.
         A Windows user account with administrative access to the SQL Server instance for the Endpoint. This user must be used to
         run the Endpoint installer.
         A TFS account with TFS administrative access to install the PreEmptive Web Components.


      Operational requirements

         A Windows user account to be used by the Endpoint to access SQL Server. This user does not need to have administrator
         access to SQL Server.
            If performing a basic install, the installer will use the same account that you supply for TFS.
            If performing an advanced install, you will specify this user account explicitly.
         A Windows user account to be used as the Application Pool identity for the new Endpoint services applications created in IIS.
         A Windows user account to run the PreEmptive Analytics Aggregator Service, with permissions to:
            Write to the file system.
            Access the network.
            Create and edit work items in Team Foundation Server.
            Optional: Query and update the TFS Maximum Attachment Size setting (only members of the “Team Foundation
            Administrators” group have access to this).


      Network access requirements
         Instrumented applications must have access to the Endpoint. The default port provided by the Endpoint installer is 8000, but
         can be changed in the installer or reconfigured after install. Port 443 will be used for SSL.
         The Endpoint must have access to the SQL Server instance where the Endpoint database resides.
         The Aggregator Service must be able to access the Endpoint using the same port configured for instrumented applications.
         (See the first bullet.)
         The Aggregator Service must have access to TFS.
         The PA for TFS configuration utility must have access to the Aggregator Configuration Service (installed along with the
         Aggregator Service) via port 80.
         Visual Studio installs on individual user machines need to have access to the PA for TFS Aggregator Configuration Service on
         port 80.



Planning a Production Installation
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 8




Endpoint and Aggregator
The Endpoint and Aggregator together provide the core feature of PreEmptive Analytics for TFS: accepting exception messages,
processing those messages through the configured rules, and creating TFS work items based on the rule results. That feature is
split across two separate runtime components in order to support different network topologies, especially corporate networks that
are split into a DMZ and an internal network.

The Endpoint is designed to run in a DMZ; it is a small web service that only accepts messages and stores them in a database.
It never sends/pushes that data anywhere else, so it does not require a firewall hole to reach the internal network. It can
respond to queries (i.e. from the Aggregator) to deliver new messages to the internal network upon request. Note that it is not
necessary to run it in a DMZ; it can be run on the same machine as the Aggregator if that meets your needs.

The Endpoint must be network-reachable by your production instrumented applications. Its host/URL should be long-lived and
stable because it will be built into your instrumented production applications.

The Endpoint requires IIS and access to a SQL Server (or SQL Express) database.

The Aggregator is designed to run inside the internal network. It is a Windows service that periodically queries the Endpoint for
new data, then processes that data through the configured rules, and then creates or updates work items in TFS as needed. The
Aggregator is also responsible for configuring individual TFS projects to use PreEmptive Analytics for TFS. It creates a local time-
limited cache of the data it processes, and also caches some state-tracking data in TFS as work item attachments.

The Aggregator must have network access (for a web service call) to the Endpoint, must have network access to the TFS server,
and must run as a user that has sufficient rights to configure TFS team projects and create and update work items in those
projects. The Aggregator is also able to query and update the TFS Maximum Attachment Size setting (as a user convenience),
but only if it is running as a user with TFS administrative rights.

One Aggregator can support multiple TFS instances and multiple team projects in each instance.


Recommendations
We recommend starting with the Endpoint and Aggregator on separate hosts. If any of your applications – now or in the future –
run outside your network, the Endpoint should be in the DMZ. If not, the Endpoint can be installed inside the internal network.
In either case, it should have a stable hostname/URL, and it should usually be on a dedicated host. The Aggregator can usually
be installed on the same host as TFS, but if you have scalability or TFS performance concerns you may want to install it on its
own machine.

There are certain scenarios that might require multiple Endpoint instances and/or multiple Aggregator instances. These scenarios
should be very rare; please contact Support if you believe you might need such a topology.


Hardware recommendations

The hardware requirements for the Endpoint and Aggregator are generally modest. Our testing demonstrated that the system
can operate normally under a load of 60 envelopes per second if the Endpoint and Aggregator are on separate VMs (with the
Aggregator on the same VM as the test TFS instance) with these specifications:
User's Guide - Version 1.2
PreEmptive Analytics for TFS User Guide | 9

   Endpoint: Intel Pentium(R) D 2.80GHz, 2.00GB RAM
   Aggregator: Six-Core AMD Opteron(tm) 2439 SE 2.79GHz (2 processors), 2.00GB RAM

60 envelopes per second is a very high volume for most contexts. An envelope is a batch of messages, usually sent by the client
every 10 seconds (depending on client platform and application configuration). A typical client will send messages when the
application starts, when it shuts down, and during execution, so in the worst case an application that is only instrumented to send
exceptions will send 3 envelopes per application run. If the Endpoint is receiving 60 envelopes per second, that translates to
roughly 200 application runs per second (60 * 10 / 3).

One important variable is the number and complexity of configured rules and subscriptions. A system with many subscriptions,
each with multiple rules, will require more resources than the example provided above. It is also possible to create custom rule
types that execute arbitrary code while evaluating rules; performance in such scenarios depends heavily on the custom
implementation.

Storage requirements are typically modest, depend heavily on both the rate of message delivery and the amount of custom data
included with each exception report. To provide a baseline, a production (mobile) application that was run approximately 90,000
times, generating approximately 5,000 exceptions, with moderate custom data, used less than 100MB of storage on the
Endpoint.

Note that the Aggregator also has a configuration setting (Data Retention Days) that determines how long it should preserve its
data cache. That setting is 30 days, by default, but can be changed as described in Defining Exception Sets and Rules. The
Endpoint does not have a similar setting, but the database has stored procedures that can be manually run to clean up old
envelopes if storage size becomes an issue.

For users wishing to be cautious in large-volume environments, we recommend putting the Aggregator on a machine that is
equivalent to one of your TFS servers.


Visual Studio Components
The Visual Studio Components provide custom controls that are necessary to view the work items (incidents) created by
PreEmptive Analytics for TFS from within Visual Studio. They also provide convenients features for accessing those work items
and for viewing and editing project-specific rules and settings.

The Visual Studio Components must be installed in every instance of Visual Studio that will be used to access work items created
by PreEmptive Analytics for TFS.

Note that Visual Studio 2012 and 2013 come pre-installed with the Visual Studio Components from the Community Edition (CE) of
PreEmptive Analytics for TFS. Installing the Pro version of these components will uninstall the CE version automatically. Please
ensure you do so, to avoid version conflicts between CE Visual Studio Components and a Pro Aggregator.

Please see Installing the Visual Studio Components for instructions.


Web Components
The Web Components provide a custom control that is necessary to view work items created by PreEmptive Analytics for TFS from
within TFS Web Access. The Aggregator installer will put the Web Components on that same machine, but from there they must
be manually installed into TFS Web Access. (This is a limitation of TFS Web Access.) See Installing the Web Components for
instructions.


Reports

The Reports do not have to be administratively installed. They will be deployed to Reporting Services automatically each time the
configuration (admin) utility is used to configure a team project to work with PreEmptive Analytics for TFS, so long as the TFS
instance is configured to use Reporting Services.
PreEmptive Analytics for TFS User Guide | 10

Performing a Basic (single-machine) Installation


      What to Expect
      When performing a Basic installation, all components will be installed to the same machine (which must be running
      Microsoft Team Foundation Server). The PreEmptive Analytics Aggregator service will be installed and set to start
      automatically. A new database will be created in your local instance of SQL Server to store exception report data
      and a new website will be created in IIS to contain the PreEmptive Analytics endpoint message & query services
      (which receive exception report messages from instrumented applications and provide access to stored messages).

         The installation must be started by a user with administrative access to the local instance of SQL Server so that
         the new database can be deployed.
         You will be asked to provide credentials for a user account with permissions to create and edit work items in Team
         Foundation Server.
         This account must also be able to log on to the local SQL Server installation and will be used as the Application
         Pool identity for the new endpoint services applications created in IIS.


      Aggregator Service Configuration
      On this screen, specify the account name and password for an account that will be used to run the PreEmptive
      Analytics aggregator service and endpoint services. This account must have permissions to create and edit work items
      in the local installation of Team Foundation Server. Refer to the Team Foundation Server Administration Console to
      locate or create an account with these privileges. This account must also be able to log on to the local SQL Server
      installation, and will be used as the Application Pool identity for the new endpoint web service applications created in
      IIS. The installer will automatically assign this account the necessary rights to access the newly-created exceptions
      database and to host the endpoint web services.


      Installation Complete
      Upon successful installation, the installer will provide you with the URL of your exception message endpoint. You will
      need to provide this endpoint when using the PreEmptive Analytics API or when instrumenting your applications for
      exception reporting with PreEmptive Dotfuscator or DashO. Instrumented applications must be able to access this
      endpoint, so you may need to adjust firewall settings to allow your target applications to send data to this endpoint.
      For more information about instrumenting your applications, see the Adding Exception Reporting to an
      Application topic.

      Once you close the installer, the PreEmptive Analytics configuration utility will launch to allow you to provision team
      projects for PreEmptive Analytics and to define rules and subscriptions to guide the creation of Incident work items.
      For more information, see the Configuring PreEmptive Analytics for TFS topic.


      Visual Studio Components
      The Visual Studio Components provide custom controls that are necessary to view the work items (incidents) created
      by PreEmptive Analytics for TFS from within Visual Studio. They also provide convenients features for accessing those
      work items and for viewing and editing project-specific rules and settings.

      The Visual Studio Components must be installed in every instance of Visual Studio that will be used to access work
      items created by PreEmptive Analytics for TFS.

      Note that Visual Studio 2012 and 2013 come pre-installed with the Visual Studio Components from the Community
      Edition (CE) of PreEmptive Analytics for TFS. Installing the Pro version of these components will uninstall the CE
      version automatically. Please ensure you do so, to avoid version conflicts between CE Visual Studio Components and
      a Pro Aggregator.

      Please see Installing the Visual Studio Components for instructions.
PreEmptive Analytics for TFS User Guide | 11

      Upload the Web Components (optional)
      In order for users to view Incident work items in Team Foundation Server Web Access, a TFS administrator must
      upload and enable the PreEmptive Analytics Web Components. Follow the instructions in the Installing the Web
      Components Components topic to enable this functionality for your users.



Performing an Advanced Installation

      When performing a Advanced installation, you choose whether to install only the endpoint services, only the aggregator
      service, or both the endpoint and aggregator services.

         When Installing the Endpoint, you choose a SQL Server instance in which to create a new database (or reuse an
         existing database) to store exception report messages and customize the installation of the PreEmptive Analytics
         endpoint message & query services (which receive exception report messages from instrumented applications and
         provide access to stored messages).
         When Installing the Aggregator, you customize the installation of the PreEmptive Analytics Aggregator service
         and configure it to retrieve exception report data from the PreEmptive Analytics endpoint services hosted on another
         machine.
         When Installing the Endpoint and Aggregator Together, both the PreEmptive Analytics endpoint services and the
         PreEmptive Analytics Aggregator service are installed on the local machine similar to performing a Basic installation.
         However, you will have the ability to customize each component as described above.



Installing the Endpoint


      What to Expect
      Installing the Endpoint creates a new database in an instance of SQL Server to store exception report data (or reuses an
      existing exception database) and creates a website in IIS to host the PreEmptive Analytics for TFS endpoint message &
      query services (which receive exception report messages from instrumented applications and provide access to stored
      messages).

         The installation must be started by a user with administrative access to the target instance of SQL Server so that the
         database can be deployed/reused.
         You will be asked to provide account credentials to be used as the Application Pool identity for the new Endpoint
         services applications created in IIS.
         You will choose whether to also use this account for Integrated Security authentication to the target instance of SQL
         Server or whether to provide a login and password for use with SQL Authentication.


      Endpoint IIS Configuration
      On this screen, specify the account name and password to be used as the IIS Application Pool identity for the PreEmptive
      Analytics for TFS Endpoint services. When you choose to use Integrated Security for communication between the
      endpoint services and SQL Server, this account will also be granted permissions to log on to the target instance of SQL
      Server.

      Next, specify the name of the IIS Application Pool that will be used to host the endpoint services and select the IIS
      website into which the PreEmptive Analytics endpoint services will be installed. When the website already exists, the
      endpoint services applications will be added to the root of the website and its port binding will not be modified. When the
      website does not exist, it will be created and a binding will be added to listen on the port specified.

      Finally, specify the location on disk into which the PreEmptive Analytics program files will be installed.
PreEmptive Analytics for TFS User Guide | 12

      Endpoint SQL Configuration
      On this screen, specify the hostname of the SQL Server instance in which a new database will be created, or where an
      existing exception database exists, to store exception report message data. Specify the name of the database to be
      used.

      Next, select either Integrated Security or SQL Authentication to be used for communication between the endpoint
      services and the SQL Server database. When using Integrated Security, the IIS Application Pool user account specified in
      the Exception Endpoint IIS Configuration screen will be used. The installer will create a new Integrated Security or
      SQL Authentication login on the target SQL Server instance if one does not exist and the specified login will be granted
      permission to use the newly-created database.


      Installation Complete
      At completion, the installer displays the URL of your exception message endpoint. Specify this endpoint URL when using
      the PreEmptive Analytics API or when instrumenting your applications for exception reporting with PreEmptive
      Dotfuscator or DashO. Instrumented applications must be able to reach this endpoint and you may need to adjust
      firewall settings to allow your target applications to send data to this endpoint. For more information about
      instrumenting your applications, see the Adding Exception Reporting to an Application topic.

      The installer also displays the URL of your endpoint query service. An Aggregator service running on another machine
      will communicate with this endpoint query service to retrieve exception data in order to create Incident work items.
      During the installation of the Aggregator on the remote machine, you will be asked to provide this URL. For more
      information, see the Installing the Aggregator topic.



Installing the Aggregator


      What to Expect
      Installing the Aggregator creates the PreEmptive Analytics for TFS Aggregator service and sets it to start automatically
      at Windows startup.

         You will be asked to provide credentials for a user account with permissions to create and edit work items in Team
         Foundation Server.


      Aggregator Service Configuration
      On this screen, specify the account name and password for an account that will be used to run the Aggregator service.
      This account must have permissions to create and edit work items in the Team Foundation Server. Refer to the Team
      Foundation Server Administration Console to locate or create an account with these privileges. Specify the location on
      disk into which the PreEmptive Analytics program files will be installed.

      Next, specify the URL of the endpoint query service running on another machine where the Endpoint is installed and
      running. The aggregator service will communicate with the provided endpoint query service to retrieve exception data in
      order to create Incident work items. When the Endpoint was installed, the URL was provided. For more information, see
      the Installing the Endpoint topic.


      Installation Complete
      Once you close the installer, the PreEmptive Analytics for TFS configuration utility launches. Use this utility to provision
      team projects for PreEmptive Analytics for TFS and to define rules and subscriptions to guide the creation of Incident
      work items. For more information, see the Configuring PreEmptive Analytics for TFS topic.


      Upload the Web Components (optional)
PreEmptive Analytics for TFS User Guide | 13

      To view Incident work items in TFS Web Access, a TFS administrator must upload and enable the PreEmptive Analytics
      for TFS Web Components. Follow the instructions in the Installing the Web Components topic to enable this functionality
      for your users.



Installing the Endpoint and Aggregator Together

      When installing the Endpoint and Aggregator together, all components will be installed to the same machine. The
      PreEmptive Analytics for TFS Aggregator service will be installed and set to start automatically at Windows startup.

         You will be asked to provide credentials for a user account with permissions to create and edit work items in Team
         Foundation Server
         A new database will be created in an instance of SQL Server to store exception report data (or an existing
         database reused)
         The installation must be started by a user with administrative access to the target instance of SQL Server so that
         the new database can be deployed (or the existing database reused)
         A new website will be created in IIS to contain the endpoint message & query services (which receives exception
         report messages from instrumented applications and provides access to stored messages)
         You will be asked to provide account credentials to be used as the Application Pool identity for the new endpoint
         services applications created in IIS
         You will choose whether to also use this account for Integrated Security authentication to the target instance of
         SQL Server or whether to provide a login and password for use with SQL Authentication


      Aggregator Service Configuration
      On this screen, specify the account name and password for an account that will be used to run the Aggregator
      service. This account must have permissions to create and edit work items in the Team Foundation Server. Refer to
      the Team Foundation Server Administration Console to locate or create an account with these privileges. Specify the
      location on disk into which the PreEmptive Analytics program files will be installed.


      Endpoint IIS Configuration
      On this screen, specify the account name and password to be used as the IIS Application Pool identity for the
      PreEmptive Analytics for TFS Endpoint services. When you choose to use Integrated Security for communication
      between the endpoint services and SQL Server, this account will also be granted permissions to log on to the target
      instance of SQL Server.

      Next, specify the name of the IIS Application Pool that will be used to host the endpoint services and select the IIS
      website into which the PreEmptive Analytics endpoint services will be installed. When the website already exists, the
      endpoint services applications will be added to the root of the website and its port binding will not be modified. When
      the website does not exist, it will be created and a binding will be added to listen on the port specified.


      Endpoint SQL Configuration
      On this screen, specify the hostname of the SQL Server instance in which a new database will be created to store
      exception report message data. Specify the name of the database to be used. Ensure that there is not an existing
      database with the same name on the selected instance of SQL Server.

      Next, select either Integrated Security or SQL Authentication to be used for communication between the endpoint
      services and the SQL Server database. When using Integrated Security, the IIS Application Pool user account
      specified in the Endpoint IIS Configuration screen will be used. The installer will create a new Integrated Security
      or SQL Authentication login on the target SQL Server instance if one does not exist and the specified login will be
      granted permission to use the newly-created database.
PreEmptive Analytics for TFS User Guide | 14

     Installation Complete
     At completion, the installer displays the URL of your exception message endpoint. Specify this endpoint URL when
     using the PreEmptive Analytics API or when instrumenting your applications for exception reporting with PreEmptive
     Dotfuscator or DashO. Instrumented applications must be able to reach this endpoint and you may need to adjust
     firewall settings to allow your target applications to send data to this endpoint. For more information about
     instrumenting your applications, see the Adding Exception Reporting to an Application topic.

     Once you close the installer, the PreEmptive Analytics for TFS configuration utility launches. Use this utility to
     provision team projects for PreEmptive Analytics for TFS and to define rules and subscriptions to guide the creation of
     Incident work items. For more information, see the Configuring PreEmptive Analytics for TFS topic.


     Visual Studio Components

     The Visual Studio Components provide custom controls that are necessary to view the work items (incidents) created
     by PreEmptive Analytics for TFS from within Visual Studio. They also provide convenients features for accessing those
     work items and for viewing and editing project-specific rules and settings.

     The Visual Studio Components must be installed in every instance of Visual Studio that will be used to access work
     items created by PreEmptive Analytics for TFS.

     Note that Visual Studio 2012 and 2013 come pre-installed with the Visual Studio Components from the Community
     Edition (CE) of PreEmptive Analytics for TFS. Installing the Pro version of these components will uninstall the CE
     version automatically. Please ensure you do so, to avoid version conflicts between CE Visual Studio Components and
     a Pro Aggregator.

     Please see Installing the Visual Studio Components for instructions.


     Upload the Web Components (optional)
     To view Incident work items in TFS Web Access, a TFS administrator must upload and enable the PreEmptive
     Analytics for TFS Web Components. Follow the instructions in the Installing the Web Components topic to enable this
     functionality for your users.



Securing the Components

     This guide shows how to configure PreEmptive Analytics for TFS to use authentication and encrypted connections.
     The reader is assumed to be familiar with IIS and with the PreEmptive Analytics for TFS components and architecture.


     Configuring the Endpoint and Fault Query Service for SSL
     This step configures the endpoint and Fault Query Service such that they support SSL connections. It is possible to
     configure each service independently (e.g. if you require SSL only on the endpoint but not on the Fault Query Service,
     it can be configured that way.).


     IIS Configuration

         1. Install the certificate on the IIS instance that is hosting the endpoint (directions for this are IIS version specific
            and are independent of PreEmptive Analytics for TFS—the configurer is assumed to be familiar with the
            process).
         2. Ensure the certificate is either signed by a Certificate Authority (CA) that clients trust by default or (e.g. if you
            are testing) that clients are configured to trust the certificate.
         3. Add an https binding for the "PreEmptive Analytics Endpoint" site in IIS. Use the certificate that was installed
            above.
PreEmptive Analytics for TFS User Guide | 15

    4. In IIS, modify the SSL settings for the Fault Query Service (i.e. the "analytics" webapp) and Endpoint
       ("message" webapp) as needed. For example, you can choose to require SSL for the Endpoint and the
       FaultQuery service.
    5. Verify that any firewalls are configured to allow inbound traffic on the configured ports for the configured
       protocols.


Checkpoint
If IIS is configured correctly, you should now be able to access both the endpoint and the Fault Query Service via
HTTPS URLs from a remote client browser. For example:

Going to https://[hostname]:[port]/message/Endpoint.ashx should result in “405 Method not allowed”.

Going to https://[hostname]:[port]/analytics/FaultQuery.svc should show the service page.

Additionally, instrumented applications now ought to be able to send analytics messages to the endpoint using SSL on
port 443.

Proceed to the next step to finish configuring the Fault Query Service.


Fault Query Service Configuration
Next, to configure the Fault Query Service to use Transport Mode security, update the "analytics" webapp's
web.config file:

    1. Add the following binding under the "webHttpBinding" element:

           
             
             
               
             
           

    2. Add the following endpoint under the "service" element:

           



Configuring the Aggregator to use SSL when connecting to the Fault
Query Service
This step configures the Aggregator’s Fault Query Service client to connect to the service using SSL.

    1. Stop the aggregator service.
    2. Edit the PreEmptive.Analytics.Aggregator.exe.config (located here by default: C:\Program Files
       (x86)\PreEmptive Solutions\PreEmptive Analytics for TFS\Service).
    3. Ensure there is a binding defined that uses "Transport" security mode, like this:
PreEmptive Analytics for TFS User Guide | 16

    4. Replace the endpoint in the "client" section with this:

           


    5. Start the aggregator service.


Checkpoint
If the aggregator and Fault Query Service are configured correctly, the aggregator should be connecting to the Fault
Query Service using SSL on port 443. This can be verified using a tool like Fiddler Web Debugger (which can be
configured to decrypt HTTPS traffic), or by looking at the IIS web logs for the Fault Query Service—the logged
requests from the aggregator should indicate a 202 response code.


Configuring the Fault Query Service for Authentication
This step configures the Fault Query Service to require a user name and password for client connections. It is
assumed (and highly recommended) that the Fault Query Service and Aggregator have already been configured to
use SSL. Further, it is recommended that the Fault Query Service be configured to require SSL (this can be done in
IIS).

    1. Make sure IIS has the required authentication module installed (e.g. Basic Authentication). On Windows 8,
       this is done via "Add/Remove Windows Features"
    2. In IIS manager, configure the "analytics" webapp to use the authentication you want (e.g. Basic
       Authentication)
    3. In the web config file for "analytics", update the binding to reflect the credential type you are using.

           
             
             
               
             
           

           

    4. Setup the account and credentials you want to use for accessing the service. This depends on which
       authentication mode you have chosen.


Checkpoint
If IIS and authentication is configured correctly, you should now only be able to access the fault query service from a
client browser if you are authenticated. For example, if you are using Basic Authentication, you should be prompted
for credentials when you visit:

https://[hostname]:[port]/analytics/FaultQuery.svc

Once successfully authenticated you should see the service page.
PreEmptive Analytics for TFS User Guide | 17

      Configuring the Aggregator to use Authentication for the Fault
      Query Service
      This step configures the Aggregator’s Fault Query Service client to provide credentials to the service.

          1. Stop the aggregator service.
          2. In the aggregator service’s app config file, update the binding to reflect the credential type you are using (it
             must match the type defined in the service configuration).

                 
                   
                     
                   
                 

          3. Add the credentials to the aggregator's app configuration file:

                 
                   
                   
                   ...
                 

          4. Restart the aggregator service


      Checkpoint

      If the aggregator and Fault Query Service are configured correctly, the aggregator should be connecting to the Fault
      Query Service using SSL on port 443. The aggregator should be providing credentials using the configured mode.
      This can be verified using a tool like Fiddler Web Debugger (which can be configured to decrypt HTTPS traffic), or by
      looking at the IIS web logs for the Fault Query Service—the logged requests from the aggregator should indicate a
      202 response code for authenticated, authorized clients and a 401 for unauthorized clients.


      Configuring the Aggregator’s Configuration Service to use SSL
      Not currently supported.


      Authentication for the Aggregator’s Configuration Service
      Not currently supported.



Installing the Visual Studio Components

      The Visual Studio Components provide custom controls that are necessary to view the work items (incidents) created by
      PreEmptive Analytics for TFS from within Visual Studio. They also provide convenient features for accessing those work
      items and for viewing and editing project-specific rules and settings.

      The Visual Studio Components must be installed in every instance of Visual Studio that will be used to access work items
      created by PreEmptive Analytics for TFS.

      Note that Visual Studio 2012 and 2013 come pre-installed with the Visual Studio Components from the Community
      Edition (CE) of PreEmptive Analytics for TFS. Installing the Pro version of these components will uninstall the CE version
      automatically. Please ensure you do so, to avoid version conflicts between CE Visual Studio Components and a Pro
      Aggregator.
PreEmptive Analytics for TFS User Guide | 18

     To install the Visual Studio Components:

         1. Execute the Visual Studio Components Installer (PreEmptive.Analytics.VisualStudio.exe).
         2. Accept the license agreement and click Install.
         3. Close and re-open Visual Studio.



Installing the Web Components

     In order for users to view Incident work items in Team Foundation Server Web Access, a TFS administrator must follow
     the following instructions:


     Upload the PreEmptive Analytics Web Components for Web Access
     in TFS 2012 or later
         1. Navigate to the TFS Web Access site in your browser.
         2. Click the Administer Team Foundation Server tile (or the gear icon in the top-right of the page if the tile is not
            visible).
         3. Select the Extensions tab and choose Install New.
         4. Click the Browse... button and select the PreEmptiveAnalyticsVisualizer.zip file from the Web subfolder within the
            PreEmptive Analytics installation directory (by default, [ProgramFilesFolder]\PreEmptive Solutions\PreEmptive
            Analytics).
         5. Click OK to upload and install the extension.
         6. Click the Enable button next to the PreEmptive Analytics Aggregator Visualizer extension.
         7. Click OK in the dialog prompting you to grant access to the extension.


     Upload the PreEmptive Analytics Web Components for Web Access in
     TFS 2010
         1. Browse to the the Web subfolder within the PreEmptive Analytics installation directory (by default,
            [ProgramFilesFolder]\PreEmptive Solutions\PreEmptive Analytics).
         2. Locate the files PreEmptive.Analytics.Aggregator.WebControl.10.0.dll and
            PreEmptiveAnalyticsVisualizer.wicc.
         3. Copy these two files to the "Application Tier\Web Access\Web\App_Data\CustomControls" subfolder within the
            Team Foundation Server 2010 installation directory (by default, [ProgramFilesFolder]\Microsoft Team Foundation
            Server 2010).
PreEmptive Analytics for TFS User Guide | 19

Configuring PreEmptive Analytics for TFS

      To integrate PreEmptive Analytics with your team projects, you must connect the PreEmptive Analytics for
      TFS Aggregator service to your Team Foundation Server instance and provision team projects for use with PreEmptive
      Analytics for TFS. Provisioned team projects can define subscriptions to receive exception report data for instrumented
      applications and rules to define how and when to create Incident work items from these exception reports.



Provisioning Team Projects

      The PreEmptive Analytics for TFS Aggregator service connects to your Team Foundation Server to create and modify
      Incident work items in response to configured rules which can include criteria such as the type and quantity of exception
      reports received. To begin using this functionality, launch the Configuration Utility to connect to your Team Foundation
      Server and provision your team projects for use with PreEmptive Analytics for TFS.


      Verify the aggregator service installation
          1. From the Start menu, launch the PreEmptive Analytics Configuration utility.
          2. The Aggregator Configuration URL will be displayed. If there is an error connecting to the Aggregator
             Configuration service, it must be corrected before you can proceed. See the Troubleshooting the Aggregator topic
             for steps in solving issues with the aggregator service.


      Connect to team project collections
          1. Select the Team Foundation Servers node and click the + button.
          2. Choose one of the configured servers from the dropdown or add a new Team Foundation Server by clicking the
             Servers... button.
          3. If the logged-in user does not have permissions to access the desired Team Foundation Server or team project
             collection, click the Use Different Credentials link and supply credentials with the desired permissions.
          4. Select a team project collection and click Connect.

                  Once you have created rules that reference this team project collection, you will not be able to delete it or
                  change the Name or Fully Qualified URL fields without first removing all the rules that reference it.




      Provision team projects for PreEmptive Analytics for TFS
      Team projects must be provisioned to be used with PreEmptive Analytics for TFS. In the Team Projects list, each team
      project in the collection is displayed along with its provisioning status. During provisioning, the Incident work item type
      and sample PreEmptive Analytics for TFS reports are imported into the project template. The project is also configured
      for use by the PreEmptive Analytics for TFS Visual Studio extension. To provision a team project to use PreEmptive
      Analytics for TFS, click the Apply link next to the desired team project. Modifications made during the provisioning
      process can be rolled back by selecting the Remove link next to any provisioned team project.
PreEmptive Analytics for TFS User Guide | 20

Defining Exception Sets and Rules

     Once you have provisioned a team project, you can begin defining the rules that tell PreEmptive Analytics for TFS how
     to map incoming exception information to work items in those projects. That mapping is defined by use
     of Subscriptions and Exception Sets which are configured via the PreEmptive Analytics for TFS Configuration
     Utility.

     Each subscription contains information about:

        The Company ID and Application ID of the application
        The team project (in TFS) in which the work items should be created
        The particular rule(s) that decide whether to create a new work item

     Each rule contains settings, including:

        The exceptions to match (via an exception set)
        The type of rule (e.g. "count the number of exception reports")
        The thresholds and time limits to use when evaluating the rule
        The type of work item to create (.e. 'Incident')

     Each exception set contains a list of exceptions, including:

        Name of the exception class
        Whether the exception was uncaught, caught, and/or thrown


     Create exception sets
         1. On the machine where the Aggregator Service is installed, launch the PreEmptive Analytics Configuration
            Utility from the Start menu, PreEmptive Solutions, PreEmptive Analytics program group.
         2. Select the Exception Sets node in the navigation panel.
         3. Click the + icon to add a new, empty exception set. Provide a name for this exception set in the Name field.
         4. In the Exceptions grid, add a new row for each exception type you would like to track. Fill in the fully-qualified
            exception type name (such as System.Net.WebException or java.io.FileNotFoundException) in the
            Exception Type column, and provide a name for this exception type (typically the unqualified exception type
            name) in the Exception Name column.

                 PreEmptive Analytics for TFS does not attempt to traverse the inheritance hierarchy of reported
                 exceptions. For example, adding a rule for System.ApplicationException will not automatically
                 match reported exceptions that are sub-types of System.ApplicationException.


         5. In the Exception Category column, you may choose whether this exception set should match Uncaught,
            Caught, and/or Thrown exceptions. Instrumented applications will send this category information along with
            the exception report data in order to provide context around the situation in which the exception occurred.

                 Incident work items contain exception information for only one exception category. If an exception set
                 specifies multiple exception categories, then matching exception report messages will be aggregated into
                 different Incident work items based on their exception category.


         6. Save the Exception Set changes by pressing Ctrl-S or invoking the File | Save menu action.

                 Once you have created rules that reference this exception set, you will not be able to delete it or change
                 the Name field without first removing all the rules that reference it.




     Create subscriptions (rules)
PreEmptive Analytics for TFS User Guide | 21

         1. Launch Visual Studio and navigate to the Team Explorer pane. Provisioned team projects will show a
            PreEmptive Analytics entry.
         2. Click the PreEmptive Analytics node to open the PreEmptive Analytics hub.
         3. In the Settings group, click Configure. This will launch the rule configuration window.
         4. Click on the General node in the left navigation.
         5. Select the Subscriptions node in the left navigation, and click the + icon to add a new, empty subscription.
         6. Provide a name for this subscription in the Name field.
         7. In the Identity group box, supply the Company and Application ID values you used when instrumenting your
            application to send exception reporting data. For more information on how to add and configure exception
            reporting in your applications, see Adding Exception Reporting to an Application.
         8. Leave the value of the Data Retention Days field set to the default of 30 unless you wish to change the
            duration for which the aggregator service will keep detailed data for each received exception.
         9. In the Team Foundation Server group box, optionally specify the area path you wish to use for the created
            work items in the Area Path field.
       10. In the Rules grid, you can create rules to be applied to exception reports received from the application you
           specified above. Please see Rule Types for details.

          We recommend starting with a relatively simple configuration and then adjusting that configuration as you see
          exception data come in from production applications. Determining the correct settings is primarily a matter of
          “tuning” the rules to achieve the outcome you want, and it is easiest if you start with very permissive rules (i.e.
          that create work items for every incident).

          To start this way, use the default All Exceptions exception set with the ErrorCountOverTime rule type, and a
          Threshold of 1.

          From there, you have many options:

              1. Increase the threshold and adjust the time period of the default (ErrorCountOverTime) rule
              2. Switch the default rule to the TopErrorsOverTime rule type, and tune that rule. (This is especially useful
                 if you have a high volume of exceptions.)
              3. Create additional exception sets that watch for specific exceptions, and add additional rules to the
                 subscription to fine-tune how individual exception classes are processed.
              4. Look on the Downloads page on preemptive.com for new rule types that might be useful for you.
              5. Create your own custom rule to suit your particular need. Please contact us to learn more.
              6. Any combination of the above.


     You can add additional subscriptions to process exception reporting data from other applications or from the same
     application using a different Area Path by selecting the Subscriptions node and clicking the + icon to create
     additional subscriptions.



Rule Types

     When configuring a rule in a subscription, you have one or more options for different Rule Types. PreEmptive Analytics
     for TFS comes with some Rule Types already installed. Additional Rule Types may be available for download under My
     Account on preemptive.com, or can be created to support custom behavior.


     Rule Types included with PreEmptive Analytics for TFS


     ErrorCountOverTime

     Create a work item when the number of exception occurrences exceeds a defined count in a given length of time.
PreEmptive Analytics for TFS User Guide | 22

          1. Choose the exception set you wish to use for this rule in the Exception Set dropdown. All exceptions that are
             included in this set can potentially trigger the execution of the rule.
          2. In the Time Period (Minutes) column, specify the length of time back from the current check time in which to sum
             exception report occurrences and evaluate whether the defined count has been exceeded. A work item is created
             when the threshold count is exceeded. The default is one day (1440 minutes).
          3. In the Threshold column, specify the exception occurrence count that, when exceeded in the specified time
             period, triggers work item creation.


      InstancesPerErrorOverTime

      Create a work item when the number of instances of running application sending an identical exception exceeds a
      defined threshold over a given length of time. This rule is useful for identifying particular errors that are affecting a large
      amount of users. An "instance" is identified by the "Instance ID" field sent with the exception data by instrumented client
      applications.

          1. Choose the exception set you wish to use for this rule in the Exception Set dropdown. All exceptions that are
             included in this set can potentially trigger the execution of the rule.
          2. In the Time Period (Minutes) column, specify the length of time back from the current check time in which to sum
             number of instances per exception and evaluate whether the defined count has been exceeded. A work item is
             created when the threshold count is exceeded. The default is one day (1440 minutes).
          3. In the Threshold column, specify the instance count that, when exceeded in the specified time period, triggers
             work item creation.


      TopErrorsOverTime

      Create work items for any exceptions whose exception occurrence count is in the top N occurrence counts within a
      defined time length. At least half the defined time length must elapse before any work items are created.

          1. Choose the exception set you wish to use for this rule in the Exception Set dropdown. All exceptions that are
             included in this set can potentially trigger the execution of the rule.
          2. In the Time Period (Minutes) column, specify the length of time back from the current check time in which to sum
             exception report occurrences and evaluate which exceptions have the top N exception counts. A work item is
             created for an exception when it is first evaluated as having a top N exception count. The default is one day
             (1440 minutes). Note that no work items will be created until half the time period has elapsed.
          3. In the Threshold column, specify the number of top exceptions (Top n) within the defined time period for which to
             create work items. "Top" refers to those exceptions having the greatest number of exception report occurrences.



Possible Surprises

      There are two aspects of the design of PreEmptive Analytics for TFS that might result in unexpected (but intentional)
      behavior. These aspects are described in the sections below.


      Historical Data
      PreEmptive Analytics for TFS is designed around a “subscription” model for processing data. What that means is that, in
      most cases, new subscriptions and new rules will only process data that is received after the subscription or rule is
      created. Any previously-received data will not be re-evaluated when a new subscription or rule is created.

      On the other hand, once a rule is active, it will cache any data that matches its settings and use that data as needed to
      correctly perform its function. The historical data cache is preserved as far back as the “Data Retention Days” field
      specifies, but each rule only looks as far back as its “Time Period,” which is usually less.


      Work Item Uniqueness
PreEmptive Analytics for TFS User Guide | 23

     A key feature of PreEmptive Analytics for TFS is that once a work item is created for a given exception, new instances of
     that same exception will simply update the prior work item rather than creating a new work item. This generally works
     as users expect but there are some scenarios that may be surprising.

     The key to the issue lies in defining the term “same exception” – i.e. how does PreEmptive Analytics for TFS decide
     whether a new exception report matches an old one? Some of the fields it uses can cause surprising side effects:

        Stack Trace: our intuitive understanding of whether two exceptions are the “same” would suggest that the stack
        trace is a critical part of that sameness. However, it is possible for a single line of code to cause the same exception
        type to be thrown (e.g. NullReferenceException) even though it was called from multiple sources. Each of those
        sources would represent a different stack trace so PreEmptive Analytics for TFS will treat those as different work
        items. In most cases this is the desired behavior but there are some situations where it might be preferable to create
        a single work item for all the different stack traces. PreEmptive Analytics for TFS has no way of identifying those
        cases, though, so it uses the more-informative option.
        Application Version: PreEmptive Analytics for TFS does not create a new work item for each distinct version of your
        application that generates the same exception. Instead, it creates a single work item and updates it with counts from
        all versions, assuming the stack traces remain identical across those versions. This may be undesirable if you fix a
        bug and then it crops up again (with the exact same stack trace) – no new work item will be created. PreEmptive
        Analytics for TFS will, however, update the Version field in the work item to reflect the version reported by each new
        exception report.
        Rule ID: work items are tied explicitly to the specific rule that created them. That has two possibly-unexpected
        consequences:
           First, if two rules both match the same exception, and both meet their threshold (at the same time or in
           sequence), two work items will be created. It will be clear from the work item titles that they came from two
           different rules but it may still be undesirable behavior. Depending on the rule configuration, though, the presence
           of multiple work items for the same exception could indicate that the exception is especially severe.
           Second, if a rule is deleted, any work items that were created by that rule will stop being updated. Recreating an
           exact copy of that rule will actually create new copies of each of the work items (as new exceptions come in and
           the thresholds are reached again) and begin updating those new work items from then forward.



Configuring the New Data Check Interval

     The new data check interval defines the frequency of checking for new exception reports and corresponding rule
     evaluation. The default check interval is 60 seconds.

     To change the check interval, you must use the PreEmptive Analytics Configuration Utility installed on the machine
     running the Aggregator Service. To change the check interval:

         1. On the machine where the Aggregator Service is installed, launch the PreEmptive Analytics Configuration Utility
            from the Start menu, PreEmptive Solutions, PreEmptive Analytics program group.
         2. Select the root machine node in the navigation panel.
         3. Locate the Aggregator Service Check Interval edit box and enter the number of seconds to wait between checks
            for new data. The default check interval is 60 seconds.
         4. Save the Exception Set changes by pressing Ctrl-S or invoking the File | Save menu action.



Configuring Custom Data Attachment Limits

     When an application is instrumented to send custom data along with exception reports, the custom data is attached to
     corresponding work items. Limits on the quantity of custom data attached to a work item are set for each TFS server.
     The default attachment size is 4 megabytes of compressed data and an unlimited number of rows that fit within the 4
     megabytes. To change the custom data attachment limits, use the PreEmptive Analytics Configuration Utility installed on
     the machine running the Aggregator Service.

     There are three settings that control the construction of custom data attachments.
Next part ... Cancel