Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://patents.google.com/patent/CN106331977B/en below:

CN106331977B - A kind of virtual reality panorama acoustic processing method of network K songs

Publication number

CN106331977B

CN106331977B CN201610704412.2A CN201610704412A CN106331977B CN 106331977 B CN106331977 B CN 106331977B CN 201610704412 A CN201610704412 A CN 201610704412A CN 106331977 B CN106331977 B CN 106331977B

Authority

CN

China

Prior art keywords

audio

network

earphone

transmitting terminal

processing

Prior art date

2016-08-22

Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)

Active

• Espacenet
• Global Dossier
• Discuss

• 238000003672 processing method Methods 0.000 title claims abstract description 20
• 238000012545 processing Methods 0.000 claims abstract description 175
• 238000000034 method Methods 0.000 claims abstract description 37
• 230000008569 process Effects 0.000 claims abstract description 21
• 230000008054 signal transmission Effects 0.000 claims abstract description 13
• 210000005069 ears Anatomy 0.000 claims description 14
• 240000006409 Acacia auriculiformis Species 0.000 claims description 10
• 230000004048 modification Effects 0.000 claims description 8
• 238000012986 modification Methods 0.000 claims description 8
• 238000012546 transfer Methods 0.000 claims description 8
• 230000005540 biological transmission Effects 0.000 claims description 7
• 238000000527 sonication Methods 0.000 claims description 6
• 230000001360 synchronised effect Effects 0.000 claims description 6
• 230000009466 transformation Effects 0.000 claims description 3
• 230000000694 effects Effects 0.000 abstract description 5
• 230000008859 change Effects 0.000 abstract description 3
• 238000012544 monitoring process Methods 0.000 abstract description 2
• 230000033001 locomotion Effects 0.000 description 5
• 102100023170 Nuclear receptor subfamily 1 group D member 1 Human genes 0.000 description 4
• 238000001914 filtration Methods 0.000 description 4
• 239000011159 matrix material Substances 0.000 description 4
• 241001342895 Chorus Species 0.000 description 3
• 230000009471 action Effects 0.000 description 3
• 238000006243 chemical reaction Methods 0.000 description 3
• HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical compound N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 2
• 238000010586 diagram Methods 0.000 description 2
• 230000004927 fusion Effects 0.000 description 2
• 238000001514 detection method Methods 0.000 description 1
• 230000003993 interaction Effects 0.000 description 1
• 230000002452 interceptive effect Effects 0.000 description 1
• 230000007704 transition Effects 0.000 description 1

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 
  • H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation  of the pseudo five- or more-channel type, e.g. virtual surround
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Human Computer Interaction (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Signal Processing (AREA)
• General Physics & Mathematics (AREA)
• Mathematical Physics (AREA)
• Computational Linguistics (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Multimedia (AREA)
• Stereophonic System (AREA)

The invention discloses a kind of virtual reality panorama acoustic processing method of network K songs, the virtual reality panorama acoustic processing method includes the following steps：Transmitting terminal audio collecting device acquires audio data, and sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；Second processing unit is overlapped processing；First processing units carry out calculation process；The playing device of signal transmission after processing to earphone plays out；By using sensing unit, real time monitoring network K sings change in location of the singer relative to listener's ear, the audio data of singer is handled relative to the spatial position of listener's ear using the sound of singer in real time, reaches network K song effects true to nature.

Technical field

The present invention relates to technical field of virtual reality, and in particular to a kind of virtual reality panorama sonication side of network K songs Method.

Background technology

Network K sings a kind of entertainment way for referring to move to KTV internet.The accompaniment of song, display are played on network The lyrics of song, user sing recording according to accompaniment and the lyrics.Finally audio data recording and accompanying song are mixed, Form the song that user individual sings.Other than individual sings, also two people are to the mode for more people's choruses of joining in the chorus.Network K is sung Other than meeting the needs of personal K song and a kind of important internet social application.But, the application of network K songs is inner at present Personal or more people record no sense of direction, affect the discrimination of audio data and social entertainment orientation.

When with virtual reality helmet (head-mounted display, HMD) to user's presentation content, in audio Hold and played by stereophone to user.We face the problem of improving virtual surrounding sound effect.In virtual reality applications, When playing audio content by stereophone, virtual 3D audios purpose is intended to reach a kind of effect and allows user just as with raising Listened in sound device array environment it is the same in addition as listening the sound in reality it is true.

When making virtual reality audio content, it usually needs the sound element of a variety of different directions.It comes personally in general, improving The method of sense is tracking user's head action, and sound is handled accordingly.If original sound perceived as coming from Front, after 90 degree of user's rotary head to the left, sound should be handled so that user perceives sound and comes from 90 degree of front-right.Using There are many kinds of classes for the virtual reality device of this processing mode, include the display equipment or headed tracking sensing of headed tracking Stereophone of device etc..Realizing head tracking, also there are many methods, and relatively common is to use multiple sensors.Movement passes Sensor external member generally includes accelerometer, gyroscope and magnetometric sensor.Each is sensed in terms of motion tracking and absolute direction The strong point and weakness that device has oneself intrinsic, therefore practices well is to use sensor " fusion " (sensor fusion) in the future It is combined from the signal of each sensor, generates a more accurate motion detection result.Obtaining end rotation angle Afterwards, it would be desirable to which sound is changed accordingly.The characteristics of network K is sung is that the multi-user for being distributed in diverse geographic location will Interaction participates in, and network K sings participant also various limb actions, position movement etc. while singing, and needs to be sung according to network K Singer changes the sound of singer in the spatial position of earphone, ability in real time relative to the change in location of listener's ear Network K song effects true to nature can be reached.

It can be seen that virtual reality panorama sound is very heavy for the network K audio data spatial impressions sung and social entertainmentization Will, but at present in this field still without suitable technical solution.In view of this, need a kind of effective network K songs virtual existing The solution of real panorama sound.

Invention content

It is existing to solve the purpose of the present invention is to provide a kind of virtual reality panorama acoustic processing method of network K songs Technology can not sing the problem of providing virtual reality panorama sound for network K

To achieve the above object, the present invention provides a kind of virtual reality panorama acoustic processing method of network K songs, the sides Method includes：

Audio is sung in the acquisition of transmitting terminal audio collecting device；

Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

First processing units carry out calculation process；

Second processing unit is overlapped processing；

The playing device of signal transmission that treated to earphone plays out.

Audio is sung in the transmitting terminal audio collecting device acquisition, including：

The transmitting terminal is set on the microphone apparatus that network K song singers use；

The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

The sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

Transmitting terminal and earphone are both provided with sensing unit；

Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

Using the ears line midpoint of earphone human ear as pole, pole is polar axis positive direction with auris dextra line, and setting pole is sat Mark system, sensing unit acquire polar coordinates variation of the transmitting terminal in the polar coordinate system in real time；

The sensing unit determines polar coordinates based on GPS positioning data；

The polar coordinates are sent to first processing units by the sensing unit in real time.

When background music is panorama sound, first it is overlapped by second processing unit by audio is sung with background music, so Calculation process is carried out by first processing units the audio data after the superposition is converted into binaural signal afterwards；

When background music is stereo signal, calculation process first is carried out by first processing units, audio conversion will be sung The binaural signal and background music signal are overlapped for binaural signal, then by second processing unit.

The first processing units carry out calculation process, including：

Audio data after singing audio or being superimposed is placed on 3D skies by the first processing units according to the polar coordinates Between in some direction, by sing audio or superposition after audio data be converted to binaural signal；

The method that audio data after singing audio or being superimposed is converted into binaural signal, to use HRTF (Head Related Transfer Function, head related transfer function) wave filter is handled or to use The transformation of Ambisonic sound fields is handled.

The second processing unit is overlapped processing, including：

The performance audio from specific senders with panorama sound background music is overlapped, obtains final K songs Song content；

The signal of the performance audio binaural signal from specific senders and background music is overlapped, is obtained most Whole K song song contents；

The specific senders can be one or multiple；

The performance audio of the multiple specific senders can be synchronous acquisition or asynchronous acquisition.

The playing device of treated signal transmission to the earphone plays out, including：

It is described that treated that signal is sent to server by first processing units or second processing unit, by server into Row distribution transmission；

The earphone is set on the playing device that network K song singers use；

The playing device is earphone.

The first processing units may be disposed at transmitting terminal and either be set to server or be set to earphone；Institute The second processing unit stated may be disposed at transmitting terminal and either be set to server or be set to earphone.

Audio data after singing audio or being superimposed is placed on 3D skies by the first processing units according to the polar coordinates Between in some direction, by sing audio or superposition after audio data be converted to binaural signal, further include：

The polar coordinates can sing participant by network K and artificially be set or changed on handheld mobile device, And pass through server and the polar coordinates of the artificial settings or modification are transmitted to first processing units.

Each network K song participants are both provided with transmitting terminal and earphone.

The method of the present invention has the following advantages that：Participant is sung according to network K and is distributed in diverse geographic location, multiuser interactive It participates in, network K sings participant while singing the characteristics of also various limb actions and position movement etc., single using sensing Member, real time monitoring network K sings change in location of the singer relative to listener's ear, opposite using the sound of singer in real time The audio data of singer is handled in the spatial position of listener's ear, reaches network K song effects true to nature.

Description of the drawings

The virtual reality panorama acoustic processing method flow diagram of Fig. 1 inventive networks K songs.

The polar coordinate system schematic diagram that Fig. 2 present invention is set using the ears line midpoint of earphone human ear as pole.

Specific embodiment

The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention..

Embodiment 1

It please refers to Fig.1, a kind of virtual reality panorama acoustic processing method of network K songs, the virtual reality panorama sonication Method includes the following steps：

Step S101：Audio is sung in the acquisition of transmitting terminal audio collecting device；

Step S102：Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

Step S103：Second processing unit is overlapped processing；

Step S104：First processing units carry out calculation process；

Step S105：The playing device of signal transmission that treated to earphone plays out.

Audio is sung in the transmitting terminal audio collecting device acquisition, including：

The transmitting terminal is set on the microphone apparatus that network K song singers use；

The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

The sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

Transmitting terminal and earphone are both provided with sensing unit；

Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

It please refers to Fig.2, using the line midpoint of the left ear 1 of earphone and auris dextra 2 as pole, pole is polar axis with 2 line of auris dextra Positive direction, sets polar coordinate system, and sensing unit acquires polar coordinates variation of the transmitting terminal in the polar coordinate system, the sensing in real time Unit determines polar coordinates based on GPS positioning data, such as at certain time point, and network K sings coordinate of the participant 3 in the polar coordinate system For (ρ₁,θ₁), it is (ρ that network K, which sings participant 4 in the coordinate of the polar coordinate system,₂,θ₂)；

The polar coordinates are sent to first processing units by the sensing unit in real time.

The background music is panorama sound, is first overlapped by second processing unit by audio is sung with background music, so Calculation process is carried out by first processing units the audio data after the superposition is converted into binaural signal afterwards；

The first processing units carry out calculation process, including：

The first processing units are according to the polar coordinates, some side audio data after superposition being placed in 3d space To the audio data after superposition is converted to binaural signal；

The method that the audio data by after superposition is converted to binaural signal, to use HRTF (Head Related Transfer Function, head related transfer function) wave filter handled.The selection of hrtf filter is opposite by transmitting terminal It being determined in the real time position direction of earphone human ear direction, the binaural signal handled is represented with B,

B=HS

Wherein, H represents HRTF filtering matrixs, and S represents the audio data of acquisition, and S here can be represented to drill corresponding to multiple Audio data after the superposition for the person of singing.

The second processing unit is overlapped processing, including：

The performance audio from specific senders with panorama sound background music is overlapped, obtains final K songs Song content；

The specific senders can be one or multiple；

The performance audio of the multiple specific senders can be synchronous acquisition or asynchronous acquisition.

The playing device of treated signal transmission to the earphone plays out, including：

It is described that treated that signal is sent to server by first processing units or second processing unit, by server into Row distribution transmission；

The earphone is set on the playing device that network K song singers use；

The playing device is earphone.

The first processing units are set to transmitting terminal, and second processing unit is set to transmitting terminal；Or first processing Unit is set to transmitting terminal, and second processing unit is set to server；Or first processing units are set to transmitting terminal, at second Reason unit is set to earphone；Or first processing units are set to server, second processing unit is set to transmitting terminal；Or First processing units are set to server, and second processing unit is set to server；Or first processing units are set to service Device, second processing unit are set to earphone；Or first processing units are set to earphone, second processing unit is set to hair Sending end；Or first processing units are set to earphone, second processing unit is set to server；Or first processing units are set Earphone is placed in, second processing unit is set to earphone.

The first processing units are according to the polar coordinates, some side audio data after superposition being placed in 3d space To the audio data after superposition is converted to binaural signal, is further included：

The polar coordinates can sing participant by network K and artificially be set or changed on handheld mobile device, And pass through server and the artificial settings and modification are transmitted to first processing units.

Each network K song participants are both provided with transmitting terminal and earphone.

Embodiment 2

It please refers to Fig.1, a kind of virtual reality panorama acoustic processing method of network K songs, the virtual reality panorama sonication Method includes the following steps：

Step S101：Audio is sung in the acquisition of transmitting terminal audio collecting device；

Step S102：Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

Step S103：Second processing unit is overlapped processing；

Step S104：First processing units carry out calculation process；

Step S105：The playing device of signal transmission that treated to earphone plays out.

Audio is sung in the transmitting terminal audio collecting device acquisition, including：

The transmitting terminal is set on the microphone apparatus that network K song singers use；

The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

The sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

Transmitting terminal and earphone are both provided with sensing unit；

Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

It please refers to Fig.2, using the line midpoint of the left ear 1 of earphone and auris dextra 2 as pole, pole is polar axis with 2 line of auris dextra Positive direction, sets polar coordinate system, and sensing unit acquires polar coordinates variation of the transmitting terminal in the polar coordinate system, the sensing in real time Unit determines polar coordinates based on GPS positioning data, such as at certain time point, and network K sings coordinate of the participant 3 in the polar coordinate system For (ρ₁,θ₁), it is (ρ that network K, which sings participant 4 in the coordinate of the polar coordinate system,₂,θ₂)；

The polar coordinates are sent to first processing units by the sensing unit in real time.

The background music is panorama sound, is first overlapped by second processing unit by audio is sung with background music, so Calculation process is carried out by first processing units the audio data after the superposition is converted into binaural signal afterwards；

The first processing units carry out calculation process, including：

The first processing units are according to the polar coordinates, some side audio data after superposition being placed in 3d space To the audio data after superposition is converted to binaural signal；

The method that the audio data by after superposition is converted to binaural signal, to become swap-in using Ambisonic sound fields Row processing.Audio data after superposition is converted into acoustic field signal, then acoustic field signal is converted to virtual speaker array signal, Virtual speaker array signal by hrtf filter is filtered, obtains binaural signal, the binaural signal B handled It represents,

B=HDTS

H represents HRTF filtering matrixs,

D represents sound field decoding matrix,

T represents sound field transition matrix,

S represents the audio data after superposition, and S here can represent the audio data from multiple and different singers.

This processing mode is advantageous in that, when the number of chorus is more, the efficiency of this processing mode is higher.

The second processing unit is overlapped processing, including：

The performance audio from specific senders with panorama sound background music is overlapped, obtains final K songs Song content；

The specific senders can be one or multiple；

The performance audio of the multiple specific senders can be synchronous acquisition or asynchronous acquisition.

The playing device of treated signal transmission to the earphone plays out, including：

It is described that treated that signal is sent to server by first processing units or second processing unit, by server into Row distribution transmission；

The earphone is set on the playing device that network K song singers use；

The playing device is earphone.

The first processing units are set to transmitting terminal, and second processing unit is set to transmitting terminal；Or first processing Unit is set to transmitting terminal, and second processing unit is set to server；Or first processing units are set to transmitting terminal, at second Reason unit is set to earphone；Or first processing units are set to server, second processing unit is set to transmitting terminal；Or First processing units are set to server, and second processing unit is set to server；Or first processing units are set to service Device, second processing unit are set to earphone；Or first processing units are set to earphone, second processing unit is set to hair Sending end；Or first processing units are set to earphone, second processing unit is set to server；Or first processing units are set Earphone is placed in, second processing unit is set to earphone.

The first processing units are according to the polar coordinates, some side audio data after superposition being placed in 3d space To the audio data after superposition is converted to binaural signal, is further included：

The polar coordinates can sing participant by network K and artificially be set or changed on handheld mobile device, And pass through server and the artificial settings and modification are transmitted to first processing units.

Each network K song participants are both provided with transmitting terminal and earphone.

Embodiment 3

It please refers to Fig.1, a kind of virtual reality panorama acoustic processing method of network K songs, the virtual reality panorama sonication Method includes the following steps：

Step S101：Audio is sung in the acquisition of transmitting terminal audio collecting device；

Step S102：Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

Step S104：First processing units carry out calculation process；

Step S103：Second processing unit is overlapped processing；

Step S105：The playing device of signal transmission that treated to earphone plays out.

Audio is sung in the transmitting terminal audio collecting device acquisition, including：

The transmitting terminal is set on the microphone apparatus that network K song singers use；

The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

The sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

Transmitting terminal and earphone are both provided with sensing unit；

Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

It please refers to Fig.2, using the line midpoint of the left ear 1 of earphone and auris dextra 2 as pole, pole is polar axis with 2 line of auris dextra Positive direction, sets polar coordinate system, and sensing unit acquires polar coordinates variation of the transmitting terminal in the polar coordinate system, the sensing in real time Unit determines polar coordinates based on GPS positioning data, such as at certain time point, and network K sings coordinate of the participant 3 in the polar coordinate system For (ρ₁,θ₁), it is (ρ that network K, which sings participant 4 in the coordinate of the polar coordinate system,₂,θ₂)；

The polar coordinates are sent to first processing units by the sensing unit in real time.

The background music is stereo signal, first carries out calculation process by first processing units, will sing audio conversion For binaural signal；The binaural signal is overlapped with background music by second processing unit again.

The first processing units carry out calculation process, including：

Performance audio is placed on some direction in 3d space, will drilled by the first processing units according to the polar coordinates It sings audio and is converted to binaural signal；

The method that performance audio is converted into binaural signal, to use HRTF (Head Related Transfer Function, head related transfer function) wave filter by sing audio be converted to four road ears (Quad binaural) signal, because To handle the binaural signal in 4 directions, thus sing audio will hrtf filter corresponding with 4 directions be all filtered. Handle obtained binaural signal B_iIt represents,

B_i=H_i·S

Wherein：I=1~N,

H_iRepresent the HRTF filtering matrixs on the i-th tunnel, S represents the performance audio of input.

The second processing unit is overlapped processing, including：

The performance audio binaural signal from specific senders with background music signal is overlapped, is obtained final K song song contents, the signal after superposition represents with B',

B'=B+M

B represents to sing audio binaural signal, M expression background music signals.

The specific senders can be one or multiple；

The performance audio of the multiple specific senders can be synchronous acquisition or asynchronous acquisition.

The playing device of treated signal transmission to the earphone plays out, including：

It is described that treated that signal is sent to server by first processing units or second processing unit, by server into Row distribution transmission；

The earphone is set on the playing device that network K song singers use；

The playing device is earphone.

The first processing units are set to transmitting terminal, and second processing unit is set to transmitting terminal；Or first processing Unit is set to transmitting terminal, and second processing unit is set to server；Or first processing units are set to transmitting terminal, at second Reason unit is set to earphone；Or first processing units are set to server, second processing unit is set to transmitting terminal；Or First processing units are set to server, and second processing unit is set to server；Or first processing units are set to service Device, second processing unit are set to earphone；Or first processing units are set to earphone, second processing unit is set to hair Sending end；Or first processing units are set to earphone, second processing unit is set to server；Or first processing units are set Earphone is placed in, second processing unit is set to earphone.

Audio data after singing audio or being superimposed is placed on 3D skies by the first processing units according to the polar coordinates Between in some direction, by sing audio or superposition after audio data be converted to binaural signal, further include：

The polar coordinates can sing participant by network K and artificially be set or changed on handheld mobile device, And pass through server and the artificial settings and modification are transmitted to first processing units.

Each network K song participants are both provided with transmitting terminal and earphone.

Embodiment 4

It please refers to Fig.1, a kind of virtual reality panorama acoustic processing method of network K songs, the virtual reality panorama sonication Method includes the following steps：

Step S101：Audio is sung in the acquisition of transmitting terminal audio collecting device；

Step S102：Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

Step S104：First processing units carry out calculation process；

Step S103：Second processing unit is overlapped processing；

Step S105：The playing device of signal transmission that treated to earphone plays out.

Audio is sung in the transmitting terminal audio collecting device acquisition, including：

The transmitting terminal is set on the microphone apparatus that network K song singers use；

The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

The sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

Transmitting terminal and earphone are both provided with sensing unit；

Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

It please refers to Fig.2, using the line midpoint of the left ear 1 of earphone and auris dextra 2 as pole, pole is polar axis with 2 line of auris dextra Positive direction, sets polar coordinate system, and sensing unit acquires polar coordinates variation of the transmitting terminal in the polar coordinate system, the sensing in real time Unit determines polar coordinates based on GPS positioning data, such as at certain time point, and network K sings coordinate of the participant 3 in the polar coordinate system For (ρ₁,θ₁), it is (ρ that network K, which sings participant 4 in the coordinate of the polar coordinate system,₂,θ₂)；

The polar coordinates are sent to first processing units by the sensing unit in real time.

The background music is stereo signal, first carries out calculation process by first processing units, will sing audio conversion For binaural signal；The binaural signal is overlapped with background music by second processing unit again.

The first processing units carry out calculation process, including：

Performance audio is placed on some direction in 3d space, will drilled by the first processing units according to the polar coordinates It sings audio and is converted to binaural signal；

The method that performance audio is converted to binaural signal, will to be handled using the transformation of Ambisonic sound fields The performance audio of acquisition is converted to the acoustic field signal in N=4 direction, then acoustic field signal is converted to virtual speaker array letter Number, virtual speaker array signal by hrtf filter is filtered, obtains the binaural signal B in N=4 direction_i,

B_i=H_i·D_i·R_i·S_Amb

Wherein：I=1~N,

H_iRepresent the HRTF filtering matrixs on the i-th tunnel,

D_iRepresent the decoding matrix on the i-th tunnel,

R_iRepresent the spin matrix on the i-th tunnel,

S_AmbRepresent the audio sound field of input.

The second processing unit is overlapped processing, including：

According to transmitting terminal relative to the real time position of earphone human ear direction, by four road binaural signals into row interpolation, Panorama acoustic field signal B is restored,

Wherein：I=1~N, G_iRepresent the interpolation coefficient on the i-th tunnel.

In order to keep signal energy, cosine signal may be used as interpolation coefficient, cosine signal G_iFunction expression For,

If G_i＜ 0, then G_i=0；

Wherein：I=1~N, θ represent the level angle of number of people rotation.

The performance audio binaural signal from specific senders with background music signal is overlapped, is obtained final K song song contents, the signal after superposition represents with B',

B'=B+M

B represents to sing audio binaural signal, M expression background music signals.

The specific senders can be one or multiple；

The performance audio of the multiple specific senders can be synchronous acquisition or asynchronous acquisition.

The playing device of treated signal transmission to the earphone plays out, including：

It is described that treated that signal is sent to server by first processing units or second processing unit, by server into Row distribution transmission；

The earphone is set on the playing device that network K song singers use；

The playing device is earphone.

The first processing units are set to transmitting terminal, and second processing unit is set to transmitting terminal；Or first processing Unit is set to transmitting terminal, and second processing unit is set to server；Or first processing units are set to transmitting terminal, at second Reason unit is set to earphone；Or first processing units are set to server, second processing unit is set to transmitting terminal；Or First processing units are set to server, and second processing unit is set to server；Or first processing units are set to service Device, second processing unit are set to earphone；Or first processing units are set to earphone, second processing unit is set to hair Sending end；Or first processing units are set to earphone, second processing unit is set to server；Or first processing units are set Earphone is placed in, second processing unit is set to earphone.

Audio data after singing audio or being superimposed is placed on 3D skies by the first processing units according to the polar coordinates Between in some direction, by sing audio or superposition after audio data be converted to binaural signal, further include：

The polar coordinates can sing participant by network K and artificially be set or changed on handheld mobile device, And pass through server and the artificial settings and modification are transmitted to first processing units.

Each network K song participants are both provided with transmitting terminal and earphone.

Although above having used general explanation and specific embodiment, the present invention is described in detail, at this On the basis of invention, it can be made some modifications or improvements, this will be apparent to those skilled in the art.Therefore, These modifications or improvements without departing from theon the basis of the spirit of the present invention belong to the scope of protection of present invention.

1. A kind of 1. virtual reality panorama acoustic processing method of network K songs, which is characterized in that the virtual reality panorama sonication side Method includes：

   Audio is sung in the acquisition of transmitting terminal audio collecting device；

   Sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction；

   First processing units carry out calculation process；

   Second processing unit is overlapped processing；

   The playing device of signal transmission that treated to earphone plays out；

   Wherein, the sensing unit acquires real time position of the transmitting terminal relative to earphone human ear direction, including：

   Transmitting terminal and earphone are both provided with sensing unit；

   Data exchange is carried out by server between the sensing unit, between sensing unit and first processing units；

   Using the ears line midpoint of earphone human ear as pole, pole is polar axis positive direction with auris dextra line, sets polar coordinate system, Sensing unit acquires polar coordinates variation of the transmitting terminal in the polar coordinate system in real time；

   The sensing unit determines polar coordinates based on GPS positioning data；

   The polar coordinates are sent to first processing units by the sensing unit in real time.

2. 2. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that the transmission Audio collecting device acquisition is held to sing audio, including：

   The transmitting terminal is set on the microphone apparatus that network K song singers use；

   The transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio of number format.

3. 3. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that

   When background music is panorama sound, first it is overlapped by second processing unit by audio is sung with background music, Ran Houyou First processing units carry out calculation process and the audio data after the superposition are converted to binaural signal；

   When background music is stereo signal, first by first processing units carry out calculation process, will sing audio be converted to it is double Ear signal, then be overlapped the binaural signal and background music signal by second processing unit.

4. 4. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that described first Processing unit carries out calculation process, including：

   Audio data after singing audio or being superimposed is placed in 3d space by the first processing units according to the polar coordinates Some direction, by sing audio or superposition after audio data be converted to binaural signal；

   The method that audio data after singing audio or being superimposed is converted into binaural signal, to use HRTF (Head Related Transfer Function, head related transfer function) wave filter handled or for using Ambisonic sound Field transformation is handled.

5. 5. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that described second Processing unit is overlapped processing, including：

   Performance audio from specific senders and panorama sound background music are overlapped, obtain final K song song contents；

   Performance audio binaural signal from specific senders and the signal of background music are overlapped, obtain final K songs Song content；

   The specific senders are at least one；

   The performance audio of the specific senders is synchronous acquisition or asynchronous acquisition.

6. 6. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that the processing The playing device of signal transmission afterwards to earphone plays out, including：

   Described treated that signal is sent to server by first processing units or second processing unit, is divided by server Hair transmission；

   The earphone is set on the playing device that network K song singers use；

   The playing device is earphone.

7. 7. the virtual reality panorama acoustic processing method of network K according to claim 1 song, which is characterized in that described the One processing unit may be disposed at transmitting terminal and either be set to server or be set to earphone；The second processing unit Transmitting terminal is may be disposed at either to be set to server or be set to earphone.

8. 8. the virtual reality panorama acoustic processing method of network K songs according to claim 4, which is characterized in that described first For processing unit according to the polar coordinates, the audio data by performance audio or after being superimposed is placed on some direction in 3d space, Audio data after singing audio or being superimposed is converted into binaural signal, is further included：

   The polar coordinates are artificially set or are changed on handheld mobile device, and passed through clothes by network K song participants The polar coordinates of the artificial settings or modification are transmitted to first processing units by business device.

9. 9. the virtual reality panorama acoustic processing method of network K songs according to claim 1, which is characterized in that each network K Song participant is both provided with transmitting terminal and earphone.

• 2016
  • 2016-08-22 CN CN201610704412.2A patent/CN106331977B/en active Active

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4