FORUMS

Ultra-simple integration with the ML kit to implement word broadcasting

24 posts
Thanks Meter: 1
 
By John Wo, Official Huawei Rep on 30th June 2020, 07:12 AM
Post Reply Email Thread
More articles like this, you can visit HUAWEI Developer Forum and Medium

Background
I believe that we all start to learn a language when we have done dictation, now when the primary school students learn the language is an important after - school work is dictating the text of the new words, many parents have this experience. However, on the one hand, the pronunciation is relatively simple. On the other hand, parents' time is very precious. Now, there are many dictation voices in the market. These broadcasters record the dictation words in the language teaching materials after class for parents to download. However, this kind of recording is not flexible enough, if the teacher leaves a few extra words today that are not part of the after-school problem set, the recordings won't meet the needs of parents and children. This document describes how to use the general text recognition and speech synthesis functions of the ML kit to implement the automatic voice broadcast app. You only need to take photos of dictation words or texts, and then the text in the photos can be automatically played. The tone color and tone of the voice can be adjusted.

https://img.xda-cdn.com/bnMyLHRilpp60p0veeIZFUmvE18=/https%3A%2F%2Fcommunityfile-dre.op.hicloud.com%2FFileServer%2FgetFile%2Fcmtybbs%2F024%2F145%2F995%2F5190044000024145995.20200429075054.10040128706233039805294595890278%3A50510630065726%3A2800%3A2BB0905E0C399B8F533FE84A55F5BE323E6692771316D751F9F3499E0A78976F.jpg

Development Preparations
Open the project-level build.gradle file

https://img.xda-cdn.com/Mho0S9W5D5tSscuGA1l8t1qu9yk=/https%3A%2F%2Fcommunityfile-dre.op.hicloud.com%2FFileServer%2FgetFile%2Fcmtybbs%2F034%2F238%2F374%2F5190001000034238374.20200624063752.62398521025056406846780924853394%3A50510630055936%3A2800%3A425420EAC833B21F29D5CCFC4138C8EE4DADA6CCFC72AA1106B77968D6EFAC2A.png

Choose allprojects > repositories and configure the Maven repository address of HMS SDK.

Code:
allprojects {
    repositories {
        google()
        jcenter()
        maven {url 'http://developer.huawei.com/repo/'}
    }
 }
Configure the Maven repository address of HMS SDK in buildscript->repositories.

Code:
buildscript {
    repositories {
        google()
        jcenter()
        maven {url 'http://developer.huawei.com/repo/'}
    }
 }
Choose buildscript > dependencies and configure the AGC plug-in.

Code:
dependencies {
    classpath 'com.huawei.agconnect:agcp:1.2.1.301'
}
Adding Compilation Dependencies
Open the application levelbuild.gradle file.

https://img.xda-cdn.com/lWuAy01VT6fjd4TPoAGDG5rl2-I=/https%3A%2F%2Fcommunityfile-dre.op.hicloud.com%2FFileServer%2FgetFile%2Fcmtybbs%2F034%2F238%2F374%2F5190001000034238374.20200624063819.36555625536468755893716957164333%3A50510630055936%3A2800%3A2DF7B9E49DF4C5938C63ECD332DC7E4F5E8AC12E25C916C62F4D16BFC77D02FC.png

SDK integration

Code:
dependencies{ 
    implementation 'com.huawei.hms:ml-computer-voice-tts:1.0.4.300'
    implementation 'com.huawei.hms:ml-computer-vision-ocr:1.0.4.300'
    implementation 'com.huawei.hms:ml-computer-vision-ocr-cn-model:1.0.4.300'
}
Add the ACG plug-in to the file header.

Code:
apply plugin: 'com.huawei.agconnect'
Specify permissions and features: Declare them in AndroidManifest.xml.

Code:
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />
Key Development Steps
There are two main functions. One is to identify the operation text, and the other is to read the operation. The OCR+TTS mode is used to read the operation. After taking a photo, click the play button to read the operation.

Dynamic permission application

Code:
private static final int PERMISSION_REQUESTS = 1;
@Override
public void onCreate(Bundle savedInstanceState) {
    // Checking camera permission  
   if (!allPermissionsGranted()) {
       getRuntimePermissions();
   }
}
2. Start the reading interface.

Code:
public void takePhoto(View view) {
        Intent intent = new Intent(MainActivity.this, ReadPhotoActivity.class);
        startActivity(intent);
}
3.Invoke createLocalTextAnalyzer() in the onCreate() method to create a device-side text recognizer.

Code:
private void createLocalTextAnalyzer() {
        MLLocalTextSetting setting = new MLLocalTextSetting.Factory()
                .setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
                .setLanguage("zh")
                .create();
        this.textAnalyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting);
 
}
4.Invoke createLocalTextAnalyzer() in the onCreate() method to create a device-side text recognizer.

Code:
private void createTtsEngine() {
        MLTtsConfig mlConfigs = new MLTtsConfig()
                .setLanguage(MLTtsConstants.TTS_ZH_HANS)
                .setPerson(MLTtsConstants.TTS_SPEAKER_FEMALE_ZH)
                .setSpeed(0.2f)
                .setVolume(1.0f);
        this.mlTtsEngine = new MLTtsEngine(mlConfigs);
        MLTtsCallback callback = new MLTtsCallback() {
            @Override
            public void onError(String taskId, MLTtsError err) {
            }
 
            @Override
            public void onWarn(String taskId, MLTtsWarn warn) {
            }
 
            @Override
            public void onRangeStart(String taskId, int start, int end) {
            }
 
            @Override
            public void onEvent(String taskId, int eventName, Bundle bundle) {
                if (eventName == MLTtsConstants.EVENT_PLAY_STOP) {
                    if (!bundle.getBoolean(MLTtsConstants.EVENT_PLAY_STOP_INTERRUPTED)) {
                        Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.read_finish, Toast.LENGTH_SHORT).show();
                    }
                }
            }
        };
        mlTtsEngine.setTtsCallback(callback);
}
5.Set the buttons for reading photos, taking photos, and reading aloud.

Code:
this.relativeLayoutLoadPhoto.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
ReadPhotoActivity.this.selectLocalImage(ReadPhotoActivity.this.REQUEST_CHOOSE_ORIGINPIC);
            }
        });
this.relativeLayoutTakePhoto.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
ReadPhotoActivity.this.takePhoto(ReadPhotoActivity.this.REQUEST_TAKE_PHOTO);
            }
        });
6.Start TextAnalyzer() during the callback of photographing and reading photos.

Code:
private void startTextAnalyzer() {
        if (this.isChosen(this.originBitmap)) {
            MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();
            Task<MLText> task = this.textAnalyzer.asyncAnalyseFrame(mlFrame);
            task.addOnSuccessListener(new OnSuccessListener<MLText>() {
                @Override
                public void onSuccess(MLText mlText) {
                    // Transacting logic for segment success.
                    if (mlText != null) {
                        ReadPhotoActivity.this.remoteDetectSuccess(mlText);
                    } else {
                        ReadPhotoActivity.this.displayFailure();
                    }
                }
            }).addOnFailureListener(new OnFailureListener() {
                @Override
                public void onFailure(Exception e) {
                    // Transacting logic for segment failure.
                    ReadPhotoActivity.this.displayFailure();
                    return;
                }
            });
        } else {
            Toast.makeText(this.getApplicationContext(), R.string.please_select_picture, Toast.LENGTH_SHORT).show();
            return;
        }
}
7.After the recognition is successful, click the play button to start the playback.

Code:
this.relativeLayoutRead.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                if (ReadPhotoActivity.this.sourceText == null) {
                    Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.please_select_picture, Toast.LENGTH_SHORT).show();
                } else {
                    ReadPhotoActivity.this.mlTtsEngine.speak(sourceText, MLTtsEngine.QUEUE_APPEND);
                    Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.read_start, Toast.LENGTH_SHORT).show();
                }
            }
        });
Demo

https://img.xda-cdn.com/6_jLN4suJl1hebKvzKQfn0qHcSY=/https%3A%2F%2Fcommunityfile-dre.op.hicloud.com%2FFileServer%2FgetFile%2Fcmtybbs%2F034%2F238%2F374%2F5190001000034238374.20200624063613.61104657732468160159798668915977%3A50510630055936%3A2800%3A0274E412DE6EB5A08EAD2FA50B7CDFE527E01FD3CC386E57FAF0004747B455F2.gif

Any questions about this process, you can visit HUAWEI Developer Forum.
1st July 2020, 09:57 AM |#2  
Official Huawei Rep
Thanks Meter: 10
 
More
Seems quite simple and useful. I will try.
2nd July 2020, 05:53 AM |#3  
Junior Member
Flag Bangalore
Thanks Meter: 0
 
More
Hi,

Nice article. Can you use ML kit for scanning product and find that product online to buy.

Thanks
2nd July 2020, 02:38 PM |#4  
Junior Member
Thanks Meter: 0
 
More
Nice and useful article
Thanks for sharing!!
3rd July 2020, 02:48 AM |#5  
Official Huawei Rep
Thanks Meter: 10
 
More
Quote:
Originally Posted by sanghati

Hi,

Nice article. Can you use ML kit for scanning product and find that product online to buy.

Thanks

Hi, if you want to scan products which you want to buy, you can use scan kit. Refer the document and acquire help from HUAWEI Developer Forum.
Post Reply Subscribe to Thread

Tags
huawei ml kit; hms core; android

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes