You have two options:
1) Buy impedance adapter to set impedance of headphones to 50+ ohm
2) You're right. You can mess with audio_mixer_tavil.xml and this will set it.
Code:
<!-- ESS DAC START -->
<path name="headphones-hifi-dac">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="1" />
</path>
<path name="headphones-hifi-dac-advanced">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="2" />
</path>
<path name="headphones-hifi-dac-aux">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="3" />
</path>
<path name="headphones-hifi-dac-44.1">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="1" />
</path>
<path name="headphones-hifi-dac-44.1-advanced">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="2" />
</path>
<path name="headphones-hifi-dac-44.1-aux">
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="3" />
</path>
<path name="headphones-hifi-dacdop">
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="1" />
</path>
<path name="headphones-hifi-dacdop-advanced">
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="2" />
</path>
<path name="headphones-hifi-dacdop-aux">
<ctl name="Es9018 Master Volume" value="0" />
<ctl name="Es9018 AVC Volume" value="0" />
<ctl name="Es9018 HEADSET TYPE" value="3" />
</path>
This will set analog gain to max with any jack connection.
Problem is, in android pie in stock system you will probably won't be able to modify any file in /vendor partition. It must be "replaced" with magisk module.
Next problem is that even with replaced file, stock system will load default file. To solve it you need to open terminal app and type:
su
killall audioserver
After that you will have properly reloaded file, and volume will be maxed, but unfortunately it needs to be applied after every reboot. Oreo is ok, pie is ****ed up. That's why i am using custom rom.