Below is the test report for the eInvoicing pilot. Additional documentation is included here.

 

Table of Contents

Introduction

The test senario was to test the 4-corner model – we are using the 4-corner (C1,C2,C3,C4) setup with access points (AP). Between corner C2 and C3 the protocol for transport is AS4 with the use of Holodeck for setup of the AS4 enviroment. The installations are using Java. The access points (AP) makes lookup in the SMP for finding the access point address of the receiver AP.

There are no standards for sending document from C1 to C2 or from C3 to C4. The choise for transport was done individualy for the test. The only issue is that the received document in corner C4 is the same as document sent for corner C1.

 

 

This was not tested

  • AS2 setup
  • Combinations of Holodeck and Domibus environments on AS4
  • Combinations of Java and .NET installations

Timeline testing 

Brief Chronology of the Interoperability Test

Week 2015/42 to 2016/06 - (October 12th 2015 to February 2nd 2016): Getting compliant with PEPPOL BIS eInvoice, setting up access point AS4.

Week 2016/06 to 2016/12 - (February 2nd to March 21st): Sending messages between access points (C2-C3).

Week 2016/13 to 2016/14 – (March 28th to April 4th): Sending messages and documents to CA and EO from the access points.

Week 2016/14 to 2016/18 – (April 4th to May 2nd): Sending and receiving many messages and large files.

Week 2016/19 to 2016/19 – (May 9th to May 12th): Test report writing and consolidating the logs for the test participants. 

Glossary

Holodeck B2B (Chasquis Consulting) is open-source software for B2B messaging, implementing the OASIS specifications for ebXML Messaging version 3 and the AS4 profile. It is a Java based, small & lightweight, well documented, flexible, stand-alone AS4 gateway. It is easy to connect with and fully compliant with the AS4 standard and it runs on most platforms.

Domibus 3 (Government of Germany) is an open source AS4 Gateway created to jumpstart the uptake of AS4 in e-SENS.  

Summary

The e-SENS eInvoice pilot was started in November 2015 and finished in May 2016.
The goal of the pilot was to see if the protocol AS4 was efficient to use for sending and receiving many and large files while using the 4-corner model. The goal was also to see if the 4-corner model could be used for transporting messages for a sender to a receiver using access point for sending and receiving messages.

The general solution: Use of AS4 protocol for sending and receiving messages is acceptable in the 4-corner model. Sending many files (1.000 at the same time) and large files (up to 2GB) works when using Holodeck setup for the AS4 environment.

However, actors using Java installation, Holodeck environment for AS4 setup, have only done the testing. Domibus and .NET installations where not tested because lack of test resources.

Not testing combinations of Holodeck and Domibus environments on AS4 does this test less trustworthy.
Not testing combinations of Java and .NET installations does this test less trustworthy. 
In real life combinations will appear.

Something to make a note of in this test:

  • Limited participants
  • Limited scenarios and scope
  • Low volumes
  • High incentive among participants
  • Custom solutions
  • Testing what works
  • Motivation to demonstrate positive conclusion

Pilot test of e-invoice for e-SENS

Standards used:        

  • §  AS2
  • AS4 version
  • Holodeck version
  • §  Domibus version 3
  • JAVA
  • §  .NET
  • PEPPOL BIS format profile 4A e-invoice only

Test participants

The whole test group tested C2-C3:

  • Austria (Brz - Government of Austria)
  • Czech Republic (ViaSky)
  • Denmark (IBM)
  • Holland (ChasQuis)
  • Greece (UPRC)
  • Norway (Difi)
  • Slovenia (Setcce)

Smaller group of testers tested C1-C2-C3-C4, 1000 messages sent simultaneously, large files up to 2GB:
  • Denmark (IBM)
  • Holland (ChasQuis)
  • Greece (UPRC)
  • Slovenia (Setcce)

Cost for installing AS4 with Holodeck:
  • Slovenia (Setcce)

Test log from the participants

  • Test log from the participants are noted in Annex 1(Excel spreadsheet) in this document.

Pilot Scenario Goals and Scope

eProcurement

eInvoicing

Result from the test

Comments to the tests

Goal ID

Goal Name

Goal Description

Scope statements

Fulfilled
Partly fulfilled
Not fulfilled

 

G5.1-UC4-1

Secure interoperability

Secure interoperability between access points using AS4 in the eProcurement area.

All access points must be able to send and receive invoices using AS4.

Fulfilled

All access points managed to send and receive invoices using AS4 with the Holodeck setup.

G5.1-UC4-2

Secure low cost for setup

Find the difference in set up cost, for access points with AS4 and AS2 protocols.

Access points (service providers) must document cost for setting up services.

Partly fulfilled

Date missing from participants. Difi and IBM have done this. Greece?

G5.1-UC4-3

Upgrading cost

Find cost for access point upgrading from AS2 to AS4.

Access points must document the cost of upgrading from AS2 to AS4

Not fulfilled

Date missing from participants. Slovenia has done this.

G5.1-UC4-4

Secure high volume transactions

Will AS4 protocol be an alternative for sending many invoices for a big invoice issuer?

Test for sending many PEPPOL BIS documents from an access point to another access point. The EO must be able to send many invoices through an access point. The access point must send the invoices to the receiver in reasonable time.

Fulfilled

Tested sending 100 and 1000 messages simultaneously.

It took 5 minutes to send 1000 messages between the access points with AS4.

The setup of the servers are the limitations for sending many files simultaneously.  AS4 worked without problems. 

G5.1-UC4-5

Secure large invoice transactions

Will AS4 protocol be an alternative for sending large invoices (up to 2 GB) invoice issuer?

Test for sending large files form an issuer’s access point to a receiver’s access point.

 

Fulfilled

Tested sending 100MB, 1GB and 1,5GB with AS4 and Holodeck implementation.
2GB files where tested successfully after some configuration in Holodeck.

 

User stories

eProcurement

eInvoicing

Result from the test

Comments to the tests

Example ID

Example Description

Fulfilled
Partly fulfilled
Not fulfilled

 

KE5.1-UC4-1

A subcontractor and supplier (EO) to a CA, both located within the EU, are doing business together. They already went through a contracting and award scenario such as an electronic tendering. After creating a purchase order electronically and sending it from the CA to the EO, the EO can now send its invoices to the CA. The CA is able to receive and process these invoices electronically.

Fulfilled

The EO and CO has established interoperability with another with the use of access points. The test access points are using AS4 with Holodeck setup for transmitting documents between the access points.

SMP are setup so that document type a receiver’s access point electronic address is available.

Test result: Ok

Actors that has tested:

  • All actors were compliant with AS4.
  • All actors were compliant with Holodeck setup.
  • All actors has their own SMP or has the possibility to use one of the others SMP (Slovenia uses Greece SMP).

KE5.1-UC4-2

The CA must also be able to receive electronic invoices from all other suppliers that are able to send and process these invoices electronically.

Fulfilled

All access points had the possibility to send to all other access points thru the access point infrastructure by using the SMP for finding the receivers access point electronic address.

Test result: Ok

Actors that has tested:

  • Denmark has sent documents to CA/EO (name).
  • Greece has sent documents to CA/EO (name).
  • Slovenia has send and received documents from CA/EO (name).

KE5.1-UC4-3

Multiple service providers of ERP systems across Europe deliver their systems to either CAs or EOs who are suppliers and subcontractors to the CAs. The ERP provider must then be able to produce, send and receive electronic invoices on behalf of both EOs and CAs.

Partly fulfilled

There has been no contact with ERP providers in these test cases but ERP providers are ready and this is proven though the broad use of ERP systems connected to an access point in the PEPPOL community.

KE5.1-UC4-4

The different service providers of ERP systems across Europe will have to set up an access point, which must be able to receive and distribute electronic invoices to the receiver.

Partly fulfilled

There are no standards for sending or receiving documents between EO its access point and the same for CO and its access point. There has been sent documents to EO and CO - to and from there access points using different transfer methods such as file transfer and Rest.

Test result: Ok

Actors that has tested:

  • Denmark has sent documents to CA/EO (name).
  • Greece has sent documents to CA/EO (name).
  • Slovenia has send and received documents from CA/EO (name).

KE5.1-UC4-5

The EO should then be able to send bulks of electronic invoices with the range of 100 and 1.000 electronic invoices to one or several CAs.

Partly Fulfilled

Same conclusion as KE5.1-UC4-4

KE5.1-UC4-6

The service provider’s access point should be able to send or receive 100 and 1.000 electronic invoices to one or several CAs.

Fulfilled

There has been sent 100 and 1000 documents simultaneously from EO access point to other access points without failures.

Test result: Ok

Actors that has tested:

  • Denmark has sent documents to CA/EO (name).
  • Greece has sent documents to CA/EO (name).
  • Slovenia has send and received documents from CA/EO (name).

KE5.1-UC4-7

Some EOs would have a need to attach larger amounts of text or images to their electronic documents for the CAs and hence an EO should be able to send an electronic invoice at size 10 MB, 100 MB and up to 2 GB.

Fulfilled

There has been sent electronic invoice at size 10 MB, 100 MB 2 GB from EO’s access point to other access points.

Test result: Ok

Actors that has tested:

  • Denmark has sent documents to CA/EO (name).
  • Slovenia has send and received documents from CA/EO (name).

KE5.1-UC4-8

An EO needs to be able to produce eInvoices at size 10 MB, 100 MB and up to 2 GB and send them thru there access point to CAs access point.

Fulfilled

There has been sent electronic invoice at size 10 MB, 100 MB 2 GB from EO’s access point to other access points

Test result: Ok

Actors that has tested:

  • Denmark has sent documents to CA/EO (name).
  • Slovenia has send and received documents from CA/EO (name).

 

References

Document type to be sent:  PEPPOL 4A - use - 20140301-PEPPOL_BIS_4A-400-AppendixA-Use Case test files.zip - PEPPOL Invoice only 4A - https://joinup.ec.europa.eu/svn/peppol/PostAward/InvoiceOnly4A/

AS2 setup

http://www.peppol.eu/ressource-library/technical-specifications/infrastructure-resources  

AS4 setup

e-SENS building block - http://www.esens.eu/content/e-delivery

Service location SML(BDXR) setup
e-SENS building block: http://www.esens.eu/content/e-delivery

Capability lookup SMP setup
e-SENS building block:
http://www.esens.eu/content/e-delivery
http://www.peppol.eu/ressource-library/technical-specifications/infrastructure-resources

Holodeck setup (Chasquis Consulting)

e-SENS building block - http://www.esens.eu/content/e-delivery

Domibus 3 setup (Government of Germany) - e-SENS building block

e-SENS building block - http://www.esens.eu/content/e-delivery

  • No labels