Inter-Operability Testing (IOT) Process Flow

I have been asked couple of times about the IOT process, how it works, etc. The above picture is from a Huawei Presentation in "The 3GPP release 8 IMS Implementation, Deployment & Testing workshop".

You can read more about 3G/4G testing from my old article here.

The 3GPP release 8 IMS Implementation, Deployment & Testing workshop

The 3GPP release 8 IMS Implementation, Deployment & Testing workshop took place in Sophia Antipolis on 24-25 November 2010.

The event was attended by 70 delegates actively participating to the discussions.

Presenting companies included: Tel : A1 Telekom Austria, Alcatel Lucent, Codenomicon, Conformiq, Eircom, Elvior, ETSI, France Telecom, GSMA, Huawei, Huawei, Mobitel, NTT DoCoMo, SFR, Telecom Italia, TestingTech, TU Berlin, Wind, Wipro, ZTE.

Here are the highlights from the ETSI document:

Goals and Outcome for this workshop

Share exprience from IMS implementation

Highlight areas for further specifications, for

Standards and Testing

Learn of issues and possible resolutions

Comments from The IMS Network Testing Group

Develop IMS core network test specifications based upon 3GPP, for:

• Interoperability

• conformance

• network integration

Hold interoperability events (IMS Plugtests)

Coordinate with other organisations such as OMA, MSF, GSMA

Implementations

• Beyond small islands, second wave to replace unscalable, unmaintenable early VoIP systems

• Implementation options - Hybrid CS-GW for transition from CS to LTE, which already has 2 million subscribers on IMS/CS-GW/RNC

• Auto provisioning - to simplify complexity

• IMS functions must be implemented in the core – not in any access network, such as LTE, and can be used for non-Voice as well

Implementing RCS (Rich Communication Suite)

• RCS trial feedback - Good feedback from 400 trial users on RCS but difficult to configure SBC

• RCS implementations should include aggregation with SNS (Social Network Services)– eg contact list from Facebook

• Most appreciated feature of RCS is: - cross-operator interworking and compatibility with ordinary phones, not just smartphones

Specific Issues and Resolutions

• FAX – Delay and Jitter issues - FTTH will solve long delays etc

• Emergency and Lawful Intercept with IMS -There are standards and developed solutions available but Currently still falls back to CS /TDM

• Data Provisioning speed is important, to achieve no service interruption.

• 3GPP II-NNI: Inter-IMS Network to Network Interface - Two levels: Solx (service with control function) and Coix (connection – a pipe for media).

• “PathFinder” Global ENUM – like DNS for phone number; It is a solution to number portability and can optimise routing

About Services

• Most issues are Beyond IMS - integrating OSS/BSS, existing systems, inter-vendors interfaces

• IMS and IN - Pity the Standards did not bring IN and IMS close together; Need iFC enhancements, like in IN; Need to support combining services

• OTT and SNS dominate growth - occupies the minds of commercial people, GSMA-like services have slowed down

• Service layer (Wipro) – Telcos want one SDP to serve all - include IMS and non-IMS services, human and non-humans on NAB, context based, and charge only what is ‘consumed’

Testing Methods, Tools and Test Beds

• Integrate Conformance checking with interoperability testing

• Automation of interoperability trace checking – it can reduce costs by more than 50 % compared to manual validation

• Independent Test Bed- available EPC playground for prototyping applications

• Protocol message customisation tool - allows changing the message and customise the flow

• Security testing tool - testing by ‘fuzzing’, 100% TTCN free – everything is already build in

• IMS is a multi vendor environment - Testing and validation must be an integral part of the deployment process

Memorable Quotes

“IMS is a Journey, not a destination” (ALU)

“SDP is almost anything” (Matjas Bericic, Mobitel)

“Voice as an app versus Voice as a Service” is a challenge (Manuel Vexler, Huawei)

“IMS is not a box, it is a network” (Matjas Bericic, Mobitel)

“global ENUM is DNS for phone numbers” (Adrian Dodd, GSMA)

“Kill with one SIP” (Ari Takanen, Codenomicon)

“ IOP is the red thread running through the entire ETSI standards development process “ (Milan Zoric, ETSI)

All documents from this workshop is available at: http://docbox.etsi.org/Workshop/2010/201011_IMSWORKSHOP/

LSTI: Job nearly done!

According to Mobile Europe magazine, LSTI has nearly completed the tasks it had been created for. The following is from the report:

LSTI said it has reached Milestones for Interoperability Development Tests (IODT), Interoperability Tests (IOT) and for Friendly Customer Trials (FCT). With these Milestones complete, LSTI said it could move to its last working phase and "finish all LTE trials in a timely manner".

Following on from the Proof of Concept (PoC) tests, which was the first testing phase of the LSTI alliance, and which was completed a year ago, LSTI has now completed all Interoperability Development Tests (IODT) for both FDD (Frequency Division Duplex) and TDD (Time-Division Duplex). Furthermore, penultimate Milestone for Interoperability Tests (IOT) has also been passed. That means that the LSTI members have proved that at least three vendors for each case are about to deliver to the market interoperability tested access network and terminal equipment.

The Core Network Interface S1 (Connection Access Network to Core) IOT is almost completed and more results are expected to be delivered soon.

“Overall these results are well aligned with the previous results shared in the LSTI PoC phase. This testing and the co-operation of the vendors and operators involved have brought forward the growth of the LTE ecosystem and enabled a accelerated commercialisation of LTE-EPC by fostering technology alignment across all parties”, said Christian Kuhlins, LSTI Activity Manager IOT, Ericsson AB. “We can now see that the telecommunications industry is about to launch LTE/SAE equipment. More and more commercial network and terminal equipment will be available on the market very soon.”

Eleven LSTI operators have set-up their LTE/EPC trials and have already delivered reports built on a common testing methodology. The Trial Group has achieved one major step in passing the “Radio Access Testing” milestone which includes: Latency, State Transition, Throughput, Cell Capacity, Mobility, Basic Quality of Service and User Experience testing domains.

LSTI said that last results are expected during the next few weeks and will be presented at the Mobile World Congress 2011.

LSTI Updates from Oct. 2010

View more presentations from Zahid Ghadialy.

Recap of Handover procedures in LTE

Interoperability in LTE

View more documents from All About 4G

The Tester's Prayer

The Tester's Prayer

Oh Lord, give us our daily bugs;

and the wisdom to find the solution or pass the blame.

Oh Lord, help us find the problems before the developers;

for if they fix the problems there may be none for us left to find.

Oh Lord, make sure that developers dont fix their problems properly;

for it gives us time to sit and relax.

Oh Lord, grant us the patience when same problem comes again and again;

for some things are better left unfixed for us to find.

Oh Lord, grant us serenity when all things work fine;

and the belief that things will soon fall in line.

Thanks to everyone who contributed and helped refine this over the years :)

MSF LTE Interoperability White Paper, Jun 2010

This white paper provides a summary of the MultiService Forum’s (MSF) Global LTE Interoperability event which took place from March 15-30, 2010.

The LTE Interoperability Event is designed to test standards compliance of Evolved Packet Core network scenarios of interest to major Service Providers, and to gauge vendor support for this technology. Building on the success of previous Global MSF Interoperability (GMI) events, the LTE Interoperability event provided the first global “real network” multi-vendor trial of the Evolved Packet Core infrastructure.

Incorporating the Evolved Packet Core defined within the Third Generation Partnership Project (3GPP) Release 8 (R8) standards, the MSF architecture introduced new access tiles to support LTE access and non-3GPP (specifically eHRPD) access to EPC. The IMS core network provided the application layer for which services may be deployed, and the binding of Quality of Service utilizing the Policy and Charging Control (PCC) for the bearer.

The event demonstrated that most of the defined LTE/EPC interfaces were mature and interoperable; however limited backwards compatibility between different implementations of 3GPP Release 8 specifications did create some issues. The fact that 3GPP does not require backward compatibility is a known limitation, but it is important to understand that this is limiting interoperability with commercially available equipment. Service providers will need to factor this into vendor selection.

Highlights of the event included:-

• Sessions were successfully established via LTE access to EPC, with creation of default and dedicated bearers with appropriate Quality of Service applied.
• An end-to-end IMS Voice over LTE session was also successfully demonstrated,
• Access to the EPC via a simulated eHRPD access was successfully tested.
• Handover between LTE and eHRPD,
• Roaming was successfully tested.

Though the essential standards are reasonably mature, the implementation of early versions of the standards within several of the available implementations of network nodes highlights the problems that can arise due to non-backwards compatibility between 3GPP releases. It is also clear that early implementations have focused initially on development of LTE access to EPC and that support for legacy access (2G/3G) to EPC is somewhat behind. Events such as the MSF LTE Interoperability event highlight these issues and prove the validity of the MSF approach to achieving multi-vendor interoperability.

MSF LTE Interoperability White Paper, Jun 2010

View more documents from Zahid Ghadialy.

This paper is available to download from here.

Conformance, Interoperability and Field testing via GCF

Delivering a “Test Once, Use Anywhere” Certification Scheme for LTE Devices

View more presentations from Zahid Ghadialy.

Agilent Demo at the LTE World Summit 2010

Here is the Agilent Demo. I have blogged about Agilent's demo from last year here.

Anritsu Demo at the LTE World Summit 2010

I have blogged about the Anritsu MD8430A here and about their RTD product here. Recently they have become first test equipment vendor in the world to achieve GCF-approved test case validations for both LTE (Long Term Evolution) RF and protocol conformance testing.

Here is their Demo from LTE World Summit 2010.

Update from LSTI Trials

Lsti mwc presentation_final

View more presentations from guest99ced7.

LTE, Conformance Testing and GCF

According to Light Reading article, LSTI results indicate that most mobile operators are still in the very early stages of testing next-gen mobile broadband Long Term Evolution (LTE) and Evolved Packet Core (EPC) technology.

Meanwhile, the System Simulator manufacturers are going strong in their LTE Conformance testing.

Anritsu Corporation announced recently that it is the first test equipment vendor in the world to achieve GCF-approved test case validations for both LTE (Long Term Evolution) RF and protocol conformance testing.

Mobile terminal manufacturers must gain GCF approval to prove that their LTE terminals satisfy the 3GPP standards, meaning that there is increasing demand for an approved conformance test environment.

At the GCF CAG#22 meeting earlier last month, Anritsu successfully gained GCF approval for an industry-leading 12 RF test cases for its ME7873L RF Conformance Test System together with additional test cases for its ME7832L Protocol Conformance Test System.

Anite and Huawei demonstrated the first TD-LTE UE protocol conformance test cases. Anite and Huawei are working together to accelerate the process of TD-LTE technology for China Mobile’s World Expo.

The new TD-LTE tests build out Anite’s comprehensive portfolio for all leading 3GPP protocol technologies, from GSM through EDGE and WCDMA to the latest HSPA+ and LTE futures. Anite blends software-only host and target test solutions for 2G, 3G and LTE technologies that allow developers to adopt a total end-to-end solution for all wireless testing needs.

Starting next month, 7 layers will offer LTE testing services in three areas: regulatory compliance, certification, and operator-specific test cases.

Aeroflex Test Solutions and 7 layers have signed an agreement to provide LTE (Long Term Evolution) testing services in support of the commercialization of LTE mobile phone networks in Japan. Aeroflex will install its 7100 LTE Digital Radio Test Set in 7 layersA' laboratory in Yokohama, Japan, greatly expanding 7 layersA' LTE regulatory compliance testing services.

With advanced LTE test equipment and services conveniently located, Japanese UE (user equipment) manufacturers and network operators can speed up testing in advance of LTE rollout.

Some tests and logs from the UE Conformance Testing of Protocols are available on the 3G4G website here.

LTE Conformance Testing Logs

I have added some LTE Conformance Testing Logs and Description on the 3G4G website at http://www.3g4g.co.uk/Lte/ConformanceTests/

Most of these initial tests have been submitted by Anite who has passed it using the LG dongle above.

Industry's first LTE Comformance test submitted for approval

Anite has submitted the first LTE test case 8.1.2.1 based on the conformance test specification 36.523-1. The test case was debugged using the LG Electronics LE03 UE.

This is in a way good news as the industry is moving forward at an amazing speed. The Release-8 of LTE was finalised in reality in March 09 (or Dec. 08 for some specs).

Anite has partnered with Agilent for the conformance testing and this release of TC's is a good way forward towards proving industry leadership.

Looking at the latest test cases that have been submitted, it seems another couple of tests 7.1.1.1 and 8.1.1.1 have been submitted as well.

People who are interested in technical details can look at the logs submitted and get the details of the messages that I have specified in the message flow earlier here.

====== Edited after post =====

Here is their press release which seems to have come after my blog :)

Anite, a global leader in testing technology for the wireless industry, and LG Electronics (LG), a global leader and technology innovator in mobile communications, today announced the successful verification of the industry’s first LTE protocol conformance test cases. Anite and LG Electronics have made the results from their groundbreaking work available to the members of the 3GPP standards body, so that the entire mobile industry may benefit from this milestone achievement.

Conformance testing is fundamental in leading-edge technologies, such as LTE, because it ensures that new handsets and data cards deliver both the applications and services anticipated by the end user and the ability to work seamlessly with existing users and networks. LG uses Anite’s LTE solution – which provides a suite of development tools for UE designers – to develop their devices in advance of LTE networks being available, ensuring these meet the industry’s rigorous certification requirements during the earliest stages of their development cycle.

The new tests build upon Anite’s comprehensive portfolio for all leading 3GPP protocol technologies from GSM through EDGE and WCDMA to the latest HSPA+ standards. Anite’s unique blend of software-only host and target test solutions for 2G, 3G and LTE technologies allows developers to adopt a total end-to-end test philosophy for all of their wireless testing needs, reducing both their time and cost to market.

"LTE device certification is essential in ensuring that next generation LTE wireless devices meet customer expectations. Working with LG is speeding the availability of the first LTE test cases to LTE developers, enabling the wireless industry to deploy the technology successfully and more quickly," said Paul Beaver, 3GPP Director, Anite. “Our customers can be confident that investing in Anite’s products will meet their conformance testing needs, maximising their test system utilisation and return on investment.”

Create your own LTE tests in 15 minutes with Anritsu’s RTD

Today we take an exclusive look in this blog inside Anritsu’s Rapid Test Designer (a.k.a. RTD) tool. RTD is a very powerful tool that can be used to design LTE tests for R&D purpose. RTD works in conjunction with Anritsu’s MD8430A. MD8430A has not very long back won the CTIA Emerging technology award.

For WCDMA/HSPA testing, Anritsu offered Protocol Test System (PTS) for R&D purpose. For LTE the basic tool is the RTD. The advantage of RTD as opposed to earlier generation PTS is that RTD is GUI based development environment that can speed up development and very little knowledge of test script development environment like TTCN-2 and TTCN-3 is required.

The RTD is a software tool that sits on top of a control PC. It controls the Signalling Tester (hardware) MD8430A. The diagram above shows the setup and connections of different components. The hardware (MD8430A) simulates Layer 1 (PHY), MAC, RLC and PDCP. The software (RTD) is used to simulate RRC and NAS.

When the tool starts up you are offered typical options as in case of any software. Once you have clicked on new test and provided it with a name, you are up and creating your own scenario.

The simple procedures already have a name defined but you can start giving meaningful names to complex procedures. Each name signifies the action it will be performing. For example Cell Configuration will be used initially to configure the parameters of the cell.

Clicking on the Cell Configuration will provide you with the possible options that can be used to configure the cells like the SFN Offsets, etc. All the channels that are necessary for a cell can be configured here.

Here is my attempt to randomly create something :)

And try sending some message like RRC Connection release. Didn’t quite work because it’s not in the right sequence but just shows the ease with which things can be done.

This is run time result of one of the test cases.

It may take few hours to get the hang of RTD, but once you understand how it works, you can start creating your own tests and scenarios at full speed. There are also example procedures available to get you started ;)

"The MD8430A is being used by LTE chipset manufacturers to ensure the quality of their products, speed time to market, and reduce design and production test costs." - Wade Hulon, Vice President and General Manager of Anritsu Company, Americas Sales Region.

You can learn more about MD8430A and RTD by following the links below:

• http://www.us.anritsu.com/downloads/files/MD8430A_EL1101.pdf
• http://www.eu.anritsu.com/files/MD8430A_LTE_E2101.pdf

Note: All the information mentioned in this post is my personal view and does not represent Anritsu's official views. Also if you manage to take few more minutes to create your own test then please do not blame me ;)

R&S bidding for LTE leadership

Free online LTE Tutorial is available at IEEE Communications Society Website, courtesy of Rohde & Schwarz. You can view the tutorial here.

Not long back, I saw R&S demo of CMW-500 with the LG UE at the LTE World Summit. R&S has also verified ETSI 3GPP LTE TTCN-3 test cases with Qasara and has shown successful interoperability testing between Qasara’s Virtual UE and Rohde & Schwarz's 3GPP LTE Virtual Tester.

There is also a LTE TDD Technology Overview available to download from their website here.

Testing UMTS protocols

Testing UMTS by Dan Fox, Anritsu

Its nearly three years since I wrote an FAQ on UMTS Testing. So when I got my hands on this book the other day, I so wanted to read it. It would be a while before I manage to go through the book in detail but my initial impression is that this book looks quite good.

Since the book deals with Protocol Testing, the testing has been grouped into three categories:

1. Integration Testing
2. Conformance Testing
3. Interoperability Testing

There is a chapter explaining each of these. The Conformance testing is of interest to me as I have been involved directly and indirectly with this for quite some years now. The book explains the process, standards required and submission of tests to GCF/PTCRB.

For those whom testing does not hold much charm, they can gain greater understanding of the concepts by reading Part II of the book. One thing I really liked in this book is that the diagrams explain the concepts very well. Rather than copying them straight from the 3GPP specifications, they have been improved and re-done by the author. Basic things like 'Dynamic TFCI selection' and 'Layer 2 transport channel processing flow for the 12.2 kbps RMC' are explained clearly using the diagrams.

There is just the right amount of detail in the chapters for Physical Layer, Layer 2 (MAC, RLC, PDCP) and Layer 3 (RRC, NAS). Further chapters show message flow sequence charts explaining things like 'setting up of speech call' and 'location updating procedure'. I have some basic sequence diagrams for message flow in the Tutorial section but the ones in the book are comparatively more detailed.

The book mainly covers UMTS, with an introduction to HSPA. It would be worthwhile to have the next edition covering LTE in detail. The main reason being that there are lots of changes in the case of LTE. The Air Interface has changed, the channels are different. The NAS messages and entities are different. UMTS (and HSPA) use TTCN-2 for testing but LTE uses TTCN-3. UMTS does not use MIMO (MIMO available for HSPA from Release 7 onwards) but LTE would generally always use MIMO.

Overall, this seems to be a useful book and I am looking forward to reading it in detail.