Research on Unvoiced Speech Communications using Smartphones and Mobiles

A startup on kickstarter is touting world's first voice mask for smartphones. Having said that Hushme has been compared to Bane from Batman and Dr. Hannibal Lecter. Good detail of Hushme at Engadget here.

This is an interesting concept and has come back in the news after a long gap. Even though we are well past the point of 'Peak Telephony' because we now use text messages and OTT apps for non-urgent communications. Voice will always be around though for not only urgent communications but for things like audio/video conference calls.

Back in 2003 NTT Docomo generated a lot of news on this topic. Their research paper "Unvoiced speech recognition using EMG - mime speech recognition" was the first step in trying to find a way to speak silently while the other party can hear voice. This is probably the most quoted paper on this topic. (picture source).

NASA was working on this area around the same time. They referred to this approach as 'Subvocal Speech'. While the original intention of this approach was for astronauts suits, the intention was that it could also be available for other commercial use. Also, NASA was effectively working on limited number of words using this approach (picture source).

For both the approaches above, there isn't a lot of recent updated information. While it has been easy to recognize certain characters, it takes a lot of effort to do the whole speech. Its also a challenge to play your voice rather than a robotic voice to the other party.

To give a comparison of how big a challenge this is, look at the Youtube videos where they do an automatic captions generation. Even though you can understand what the person is speaking, its always a challenge for the machine. You can read more about the challenge here.

A lot of research in similar areas has been done is France and is available here.

Motorola has gone a step further and patented an e-Tattoo that can be emblazoned over your vocal cords to intercept subtle voice commands — perhaps even subvocal commands, or even the fully internal whisperings that fail to pluck the vocal cords when not given full cerebral approval. One might even conclude that they are not just patenting device communications from a patch of smartskin, but communications from your soul. Read more here.

Another term used for research has been 'lip reading'. While the initial approaches to lip reading was the same as other approaches of attaching sensors to facial muscles (see here), the newer approaches are looking at exploiting smartphone camera for this.

Many researchers have achieved reasonable success using cameras for lip reading (see here and here) but researchers from Google’s AI division DeepMind and the University of Oxford have used artificial intelligence to create the most accurate lip-reading software ever.

Now the challenge with smartphones for using camera for speech recognition will be high speed data connectivity and ability to see lip movement clearly. While in indoor environment this can be solved with Wi-Fi connectivity and looking at the camera, it may be a bit tricky outdoors or not looking at the camera while driving. Who knows, this may be a killer use-case for 5G.

By the way, this is not complete research in this area. If you have additional info, please help others by adding it in the comments section.

Related links:

• Silent speech interface
• Japan: Phoning home may soon be at your fingertips as Japanese researchers develop phone which transmits sound through the hand
• Video: NASA Subvocal Speech Recognition
• Motorola patents e-tattoo that can read your thoughts by listening to unvocalized words in your throat
• Silent Communication: whispered speech-to-clear speech conversion
• Google’s AI can now lip read better than humans after watching thousands of hours of TV

5G – Beyond the Hype

Dan Warren, former GSMA Technology Director who created VoLTE and coined the term 'Phablet' has been busy with his new role as Head of 5G Research at Samsung R&D in UK. In a presentation delivered couple of days back at Wi-Fi Global Congress he set out a realistic vision of 5G really means.

Heh, I like this: network *splicing* may be more important than the slices that get bundled/integrated @TMGB #WGCLON pic.twitter.com/eppg1PJyu7

— Dean Bubley (@disruptivedean) May 10, 2017

A brief summary of the presentation in his own words below, followed by the actual presentation:

"I started with a comment I have made before – I really hate the term 5G.  It doesn’t allow us to have a proper discussion about the multiplicity of technologies that have been throw under the common umbrella of the term, and hence blurs the rationale for one why each technology is important in its own right.  What I have tried to do in these slides is talk more about the technology, then look at the 5G requirements, and consider how each technology helps or hinders the drive to meet those requirements, and then to consider what that enables in practical terms.

The session was titled ‘5G – beyond the hype’ so in the first three slides I cut straight to the technology that is being brought in to 5G.  Building from the Air Interface enhancements, then the changes in topology in the RAN and then looking at the ‘softwarisation’ on the Core Network.  This last group of technologies sets up the friction in the network between the desire to change the CapEx model of network build by placing functions in a Cloud (both C-RAN and an NFV-based Core, as well as the virtualisation of transport network functions) and the need to push functions to the network edge by employing MEC to reduce latency.  You end up with every function existing everywhere, data breaking out of the network at many different points and some really hard management issues.

On slide 5 I then look at how these technologies line up to meeting 5G requirements.  It becomes clear that the RAN innovations are all about performance enhancement, but the core changes are about enabling new business models from flexibility in topology and network slicing.  There is also a hidden part of the equation that I call out, which is that while technology enables the central five requirements to be met, they also require massive investment by the Operator.  For example you won’t reach 100% coverage if you don’t build a network that has total coverage, so you need to put base stations in all the places that they don’t exist today.

On the next slide I look at how network slicing will be sold.  There are three ways in which a network might be sliced – by SLA or topology, by enterprise customer and by MVNO.  The SLA or topology option is key to allowing the co-existence of MEC and Cloud based CN.  The enterprise or sector based option is important for operators to address large vertical industry players, but each enterprise may want a range of SLA’s for different applications and devices, so you end up with an enterprise slice being made up of sub-slices of differing SLA and topology.  Then, an MVNO may take a slice of the network, but will have it’s own enterprise customers that will take a sub-slice of the MVNO slice, which may in turn be made of sub-sub-slices of differing SLAs.  Somewhere all of this has be stitched back together, so my suggestion is that ‘Network Splicing’ will be as important as network slicing.

Slide illustrates all of this again and notes that there will also be other networks that have been sliced as well, be that 2G, 3G, 4G, WiFi, fixed, LPWA or anything else.  There is also going to be an overarching orchestration requirement both within a network and in the Enterprise customer (or more likely in System Integrator networks who take on the ‘Splicing’ role).  The red flags are showing that Orchestration is both really difficult and expensive, but the challenge for the MNO will also exist in the RAN.  The RRC will be a pinch point that has to sort out all of these device sitting in disparate network topologies with varying demands on the sliced RAN.

Then, in the next four slides I look at the business model around this.  Operators will need to deal with the realities of B2B or B2B2C business models, where they are the first B. The first ‘B’s price is the second ‘B’s cost, so the operator should expect considerable pressure on what it charges, and to be held contractually accountable for the performance of the network.  If 5G is going to claim 100% coverage, 5 9’s reliability, 50Mbps everywhere and be sold to enterprise customers on that basis, it is going to have to deliver it else there will be penalties to pay.  On the flip side to this, if all operators do meet the 5G targets, then they will become very much the same so the only true differentiation option will be on price.  With the focus on large scale B2B contracts, this has all the hallmarks of a race downwards and commoditisation of connectivity, which will also lead to disintermediation of operators from the value chain on applications.

So to conclude I pondered on what the real 5G justification is.  Maybe operators shouldn’t be promising everything, since there will be healthy competition on speed, coverage and reliability while those remain as differentiators.  Equally, it could just be that operators will fight out the consumer market share on 5G, but then that doesn’t offer any real uplift in market size, certainly not in mature developed world markets.  The one thing that is sure is that there is a lot of money to be spent getting there."

5G - Tech to business case from 3G4G

Let me know what do you think?

10 years battery life calculation for Cellular IoT

I made an attempt to place the different cellular and non-cellular LPWA technologies together in a picture in my last post here. Someone pointed out that these pictures above, from LoRa alliance whitepaper are even better and I agree.

Most IoT technologies lists their battery life as 10 years. There is an article in Medium rightly pointing out that in Verizon's LTE-M network, IoT devices battery may not last very long.

The problem is that 10 years battery life is headline figure and in real world its sometimes not that critical. It all depends on the application. For example this Iota Pet Tracker uses Bluetooth but only claims battery life of  "weeks". I guess ztrack based on LoRa would give similar results. I have to admit that non-cellular based technologies should have longer battery life but it all depends on applications and use cases. An IoT device in the car may not have to worry too much about power consumption. Similarly a fleet tracker that may have solar power or one that is expected to last more than the fleet duration, etc.

So coming back to the power consumption. Martin Sauter in his excellent Wireless Moves blog post, provided the calculation that I am copying below with some additions:

The calculation can be found in 3GPP TR 45.820, for NB-IoT in Chapter 7.3.6.4 on ‘Energy consumption evaluation’.

The battery capacity used for the evaluation was 5 Wh. That’s about half or even only a third of the battery capacity that is in a smartphone today. So yes, that is quite a small battery indeed. The chapter also contains an assumption on how much power the device draws in different states. In the ‘idle’ state the device is in most often, power consumption is assumed to be 0.015 mW.

How long would the battery be able to power the device if it were always in the idle state? The calculation is easy and you end up with 38 years. That doesn’t include battery self-discharge and I wondered how much that would be over 10 years. According to the Varta handbook of primary lithium cells, self-discharge of a non-rechargable lithium battery is less than 1% per year. So subtract roughly 4 years from that number.

Obviously, the device is not always in idle and when transmitting the device is assumed to use 500 mW of power. Yes, with this power consumption, the battery would not last 34 years but less than 10 hours. But we are talking about NB-IoT so the device doesn’t transmit for most of the time. The study looked at different transmission patterns. If 200 bytes are sent once every 2 hours, the device would run on that 5 Wh battery for 1.7 years. If the device only transmits 50 bytes once a day the battery would last 18.1 years.

So yes, the 10 years are quite feasible for devices that collect very little data and only transmit them once or twice a day.

The conclusions from the report clearly state:

The achievable battery life for a MS using the NB-CIoT solution for Cellular IoT has been estimated as a function of reporting frequency and coupling loss. 

It is important to note that these battery life estimates are achieved with a system design that has been intentionally constrained in two key respects:

• The NB-CIoT solution has a frequency re-use assumption that is compatible with a stand-alone deployment in a minimum system bandwidth for the entire IoT network of just 200 kHz (FDD), plus guard bands if needed.
• The NB-CIoT solution uses a MS transmit power of only +23 dBm (200 mW), resulting in a peak current requirement that is compatible with a wider range of battery technologies, whilst still achieving the 20 dB coverage extension objective.  

The key conclusions are as follows:

• For all coupling losses (so up to 20 dB coverage extension compared with legacy GPRS), a 10 year battery life is achievable with a reporting interval of one day for both 50 bytes and 200 bytes application payloads.
• For a coupling loss of 144 dB (so equal to the MCL for legacy GPRS), a 10 year battery life is achievable with a two hour reporting interval for both 50 bytes and 200 bytes application payloads. 
• For a coupling loss of 154 dB, a 10 year battery life is achievable with a 2 hour reporting interval for a 50 byte application payload. 
• For a coupling loss of 154 dB with 200 byte application payload, or a coupling loss of 164 dB with 50 or 200 byte application payload, a 10 year battery life is not achievable for a 2 hour reporting interval. This is a consequence of the transmit energy per data bit (integrated over the number of repetitions) that is required to overcome the coupling loss and so provide an adequate SNR at the receiver. 
• Use of an integrated PA only has a small negative impact on battery life, based on the assumption of a 5% reduction in PA efficiency compared with an external PA.

Further improvements in battery life, especially for the case of high coupling loss, could be obtained if the common assumption that the downlink PSD will not exceed that of legacy GPRS was either relaxed to allow PSD boosting, or defined more precisely to allow adaptive power allocation with frequency hopping.

I will look at the technology aspects in a future post how 3GPP made enhancements in Rel-13 to reduce power consumption in CIoT.

Also have a look this GSMA whitepaper on 3GPP LPWA lists the applications requirements that are quite handy.

Variety of 3GPP IoT technologies and Market Status - May 2017

I have seen many people wondering if so many different types of IoT technologies are needed, 3GPP or otherwise. The story behind that is that for many years 3GPP did not focus too much on creating an IoT variant of the standards. Their hope was that users will make use of LTE Cat 1 for IoT and then later on they created LTE Cat 0 (see here and here).

The problem with this approach was that the market was ripe for a solution to a different types of IoT technologies that 3GPP could not satisfy. The table below is just an indication of the different types of technologies, but there are many others not listed in here.

The most popular IoT (or M2M) technology to date is the humble 2G GSM/GPRS. Couple of weeks back Vodafone announced that it has reached a milestone of 50 million IoT connections worldwide. They are also adding roughly 1 million new connections every month. The majority of these are GSM/GPRS.

Different operators have been assessing their strategy for IoT devices. Some operators have either switched off or are planning to switch off they 2G networks. Others have a long term plan for 2G networks and would rather switch off their 3G networks to refarm the spectrum to more efficient 4G. A small chunk of 2G on the other hand would be a good option for voice & existing IoT devices with small amount of data transfer.

In fact this is one of the reasons that in Release-13 GSM is being enhanced for IoT. This new version is known as Extended Coverage – GSM – Internet of Things (EC-GSM-IoT ). According to GSMA, "It is based on eGPRS and designed as a high capacity, long range, low energy and low complexity cellular system for IoT communications. The optimisations made in EC-GSM-IoT that need to be made to existing GSM networks can be made as a software upgrade, ensuring coverage and accelerated time to-market. Battery life of up to 10 years can be supported for a wide range use cases."

The most popular of the non-3GPP IoT technologies are Sigfox and LoRa. Both these technologies have gained significant ground and many backers in the market. This, along with the gap in the market and the need for low power IoT technologies that transfer just a little amount of data and has a long battery life motivated 3GPP to create new IoT technologies that were standardised as part of Rel-13 and are being further enhanced in Rel-14. A summary of these technologies can be seen below

If you look at the first picture on the top (modified from Qualcomm's original here), you will see that these different IoT technologies, 3GPP or otherwise address different needs. No wonder many operators are using the unlicensed LPWA IoT technologies as a starting point, hoping to complement them by 3GPP technologies when ready.

Finally, looks like there is a difference in understanding of standards between Ericsson and Huawei and as a result their implementation is incompatible. Hopefully this will be sorted out soon.


Market Status:

Telefonica has publicly said that Sigfox is the best way forward for the time being. No news about any 3GPP IoT technologies.

Orange has rolled out LoRa network but has said that when NB-IoT is ready, they will switch the customers on to that.

KPN deployed LoRa throughout the Netherlands thereby making it the first country across the world with complete coverage. Haven't ruled out NB-IoT when available.

SK Telecom completed nationwide LoRa IoT network deployment in South Korea last year. It sees LTE-M and LoRa as Its 'Two Main IoT Pillars'.

Deutsche Telekom has rolled out NarrowBand-IoT (NB-IoT) Network across eight countries in Europe (Germany, the Netherlands, Greece, Poland, Hungary, Austria, Slovakia, Croatia)

Vodafone is fully committed to NB-IoT. Their network is already operational in Spain and will be launching in Ireland and Netherlands later on this year.

Telecom Italia is in process of launching NB-IoT. Water meters in Turin are already sending their readings using NB-IoT.

China Telecom, in conjunction with Shenzhen Water and Huawei launched 'World's First' Commercial NB-IoT-based Smart Water Project on World Water Day.

SoftBank is deploying LTE-M (Cat-M1) and NB-IoT networks nationwide, powered by Ericsson.

Orange Belgium plans to roll-out nationwide NB-IoT & LTE-M IoT Networks in 2017

China Mobile is committed to 3GPP based IoT technologies. It has conducted outdoor trials of NB-IoT with Huawei and ZTE and is also trialing LTE-M with Ericsson and Qualcomm.

Verizon has launched Industry’s first LTE-M Nationwide IoT Network.

AT&T will be launching LTE-M network later on this year in US as well as Mexico.

Sprint said it plans to deploy LTE Cat 1 technology in support of the Internet of Things (IoT) across its network by the end of July.

Further reading:

• An Introduction to IoT: Connectivity & Case Studies
• GSMA: Extended Coverage – GSM – Internet of Things (EC-GSM-IoT )
• 3GPP Rel-14 IoT Enhancements
• 5G Americas: LTE and 5G Technologies Enabling the Internet of Things [PDF]
• Samsung: Internet of Things - Introducing innumerable opportunities [PDF]
• The Korean Institute of Communications Information Sciences: A survey on LPWA technology: LoRa and NB-IoT
• Nokia: LTE evolution for IoT connectivity
• Carriers aim to crush LoRa, Sigfox and others