Smart Gird Energy Training Coalition

Department of Electrical and Computer Engineering A Hierarchical Game Framework for Distributive Resource Allocation in Future HetNet -----------------PH.D. THESIS DEFENSE ----------------Huaqing Zhang Advisor: Zhu Han Wireless Networking, Signal Processing and Security Lab Department of Electrical and Computer Engineering University of Houston, TX, USA Nov 2017 Overview Department of Electrical and Computer Engineering Introduction - Development of Wireless Network - Service Architecture A Hierarchical Game Framework for Heterogeneous Network

Multi-Tier Wireless Network LTE Unlicensed Fog Computing Zero-Determinant Strategy A Multi-Leader Multi-Follower Stackelberg Game A Three-Layer Hierarchical Game Future Works Conclusions 2 Department of Electrical and Computer Engineering Introduction Various Applications in Future Wireless Data service Increasing demands in mobile data traffic 3 Department of Electrical and Computer Engineering

Insight from Plants vs. Zombies Real-Time Requirement Wireless Network Task of Data Transmission Be Advanced Get Helpers Be the God 4 Department of Electrical and Computer Engineering Be Advanced Multi-Layer Wireless Network Development of Wireless Network Get Helpers

Cooperations of Multiple Networks Be the God Network Virtualization How to manage the distributive behaviors of autonomous individuals? 5 Department of Electrical and Computer Engineering Service Architecture Service Layer: Authentication, Billing, Mobility Management, Routing, Resource Layer: Spectrum, Time, Code, Power, Infrastructure Layer: Wireline Network, Base Stations, Relays, User Layer: Various Mobile Applications

6 Department of Electrical and Computer Engineering Traditional Wireless Network Service Provider Service Provider Service Provider Service Layer Service Layer Service Layer Resource Layer Resource

Layer Infrastructure Layer Infrastructure Layer End Users End Users Resource Layer Infrastructure Layer End Users 7 Department of Electrical and Computer Engineering Hierarchical Game Framework

Service Layer Service Layer Service Layer Resource Layer Resource Layer Resource Layer Static Game Sequential Game Infrastructure Layer Infrastructure Layer Mobile Users

Mobile Users Infrastructure Layer Mobile Users 8 Department of Electrical and Computer Engineering Contributions Modeled a distributive four-layer service architecture for future heterogeneous network Proposed a hierarchical game framework to analyze the distributive resource management - Zero-determinant strategy in infrastructure layer with the application of small cell network. - A multi-leader multi-follower Stackelberg game between service layer and user layer with the application of LTE Unlicensed. - A multi-layer hierarchical game in the general distributive service

architecture with the application of fog computing. Numerical analysis and simulations in practical scenarios are employed with stable and high QoS. 9 RoadMap Department of Electrical and Computer Engineering Be Advanced Multi-Layer Wireless Network Introduction Hierarchical Hierarchical Game Framework Framework Game Small Cell Network Get Helpers

Cooperations of Multiple Networks LTE Unlicensed Future Works Conclusions Be the God Network Virtualization Fog Computing 10 Small Cell Network Department of Electrical and Computer Engineering Macro-cell Small Cell Service Layer Service Layer

Resource Layer Infrastructure Layer End Users Infrastructure Layer End Users Macrocell base station Small cell base station Macrocell user Small cell user 11 Department of Electrical and

Computer Engineering Motivation Interference between macrocells and small cells Cooperations of resource sharing between macrocell and small cells Moreover, due to Weak communication signals Cheating strategies Some small cells may unexpectedly stop cooperation unilaterally. Macrocell is responsible to maintain the social welfare in high values. 12 Department of Electrical and Computer Engineering The Game of Chicken When considering the interaction between different autonomous

decision makers, we apply game theory to solve the problem. The Game of Chicken 13 The Game of Chicken Department of Electrical and Computer Engineering Player AoC: Administrator of Cooperation (Macrocell) PoC: cell) Participant of Cooperation (Small Chicken-Dare Game Strategy Chicken: Low transmit power Dare: High transmit power AoC PoC Chicken

Dare Chicken (R,R) (S,T) Dare (T,S) (P,P) Utility Revenue for both X and Y =[ ] =[ ] T>R>S>P

14 Department of Electrical and Computer Engineering Strategy Profile AoCs strategy is a=() PoCs strategy is b=() 15 Markov and Utility Department of Electrical and Computer Engineering Unnecessary to simulate the play of strategies a against b move by move, we adopt Markov process: The transition matrix The stationary vector of Markov process(implying reaching the equilibrium of the game) is ; The expected utilities of both players AoC and PoC are, respectively,

= = T T T T = = 16 Zero-Determinant

Method Department of Electrical and Computer Engineering Cramers rule = Adj ( Adjis the adjugate matrix of ) We set = = The matrix is singular, with thus zero determinant. Adj Every row of Adj is proportional to . 17 Department of Electrical and Computer Engineering

Zero-Determinant Method We do dot product of an arbitrary four-vector with the stationary vector : 1+1 1 1+1 1+1 1 2 3 1+2 3 2 =(,, )= 3 2 3 1+2 3 4 4 4 4 4 [ ~ ] ~ Notably, The second column is solely under the control of AoC;

The third column is solely under the control of PoC; The fourth column is simply . 18 Zero-Determinant Method Department of Electrical and Computer Engineering We suppose = + + Both AoC and PoC have the possibility of choosing unilateral strategies that will make the determinant in the numerator vanish. i.e. If AoC chooses a strategy that satisfies or if PoC chooses a strategy that satisfies ~ = ( + + ) Then the determinant vanishes and a linear relation between the two scores,

+ + = We call these zero-determinant (ZD) strategies. 19 Department of Electrical and Computer Engineering Constraints Social Welfare Zero-determinant Strategy Ratio Relationship Whatever the strategy of PoC, by adopting strategy , AoC is able to unilaterally maintain the social welfare in high and fixed value Optimal Strategy Maximum Social Welfare 20

Simulations Department of Electrical and Computer Engineering Energy-Saving Aggressive Tit-for-Tat Pavlov Whatever the behaviors of the other component, zero-determinant strategy is able to maintain the social welfare in fixed and high values 21 Department of Electrical and Computer Engineering Simulations

With the number of PoCs increasing, and the number of actions increasing, the maintained social welfare increases 22 RoadMap Department of Electrical and Computer Engineering Be Advanced Multi-Layer Wireless Network Introduction Hierarchical Hierarchical Game Framework Framework Game Small Cell Network Get Helpers Cooperations of Multiple Networks LTE Unlicensed Future Works

Conclusions Be the God Network Virtualization Fog Computing 23 Department of Electrical and Computer Engineering LTE Unlicensed Wireless Operator Wi-Fi Service Layer Service Layer Resource Layer Resource

Layer Infrastructure Layer Infrastructure Layer End Users End Users Integrate unlicensed carrier into the overall LTE system by adapting LTE air interface to operate in the unlicensed spectrum 24 Department of Electrical and Computer Engineering Motivation 25 Department of Electrical and Computer Engineering

Game Framework Non-cooperative Game Multi-Leader Multi-Follower Stackelberg Game Matching 26 Utility Department of Electrical and Computer Engineering User: Revenue the user j obtain for unit data transmission Binary wherenumber determine if the sub-band s in unlicensed spectrum is allocated to user j First-Mover Advantage

Transmit power of the user j in the unlicensed spectrum Price set by the operator i in the unlicensed spectrum Interference in the sub-band s of the unlicensed spectrum Operator: 27 Strategy of Users Department of Electrical and Computer Engineering Backward Induction Strategies of Users Sub-bands and Users Strategies of Operators The utility of the user j is convex. We take the first derivative with respective to

where The maximum transmit power constraint of the user j in the unlicensed spectrum 28 Gale-Shapley Algorithm Department of Electrical and Computer Engineering Stable Marriage Matching Problem Geeta, Heiki, Irina, Fran Fran Adam Irina, Fran, Heiki, Geeta Bob This is a stable matching Carl > Adam Geeta

Geeta, Fran, Heiki, Irina Carl Heiki Irina, Heiki, Geeta, Fran David David > Bob Irina 29 Sub-band Allocation Department of Electrical and Computer Engineering Adam, Bob, Carl, David Geeta, Heiki, Irina, Fran Adam Fran Carl, David, Bob, Adam

Bob USER Irina, Fran, Heiki, Geeta SUB-BAND Geeta Carl, Bob, David, Adam Geeta, Fran, Heiki, Irina Carl Heiki Adam, Carl, David, Bob David Irina, Heiki, Geeta, Fran Irina 30 Department of Electrical and Computer Engineering

Sub-band Allocation with Matching Preference Lists Man User Woman Sub-band where Algorithm Evaluation 31 Department of Electrical and Computer Engineering Non-cooperative Strategies for Operators

We put the optimal solution and sub-band allocation into the utility function of operators Subgradient Method Algorithm 32 Department of Electrical and Computer Engineering Cooperative Strategies for Operators where 33 Department of Electrical and Computer Engineering Simulation Results The utility of operators generally increases with number of UE increasing

The interference from Wi-Fi affect the performance UE and the utility of operators 34 RoadMap Department of Electrical and Computer Engineering Be Advanced Multi-Layer Wireless Network Introduction Hierarchical Hierarchical Game Framework Framework Game Small Cell Network Get Helpers Cooperations of Multiple Networks LTE Unlicensed Future Works

Conclusions Be the God Network Virtualization Fog Computing 35 Department of Electrical and Computer Engineering Cloud Computing 36 Department of Electrical and Computer Engineering Fog Computing Fog computing is an architecture that uses one or a collaborative multitude of end-user clients or near-user edge devices to carry out a substantial amount of storage, communication, and control, configuration, measurement and management. Data Center/Cloud Non-real-time data action, storage

Fog Local data analysis and filtering Device Local data collection 37 Department of Electrical and Computer Engineering System Architecture FN: Fog node DSO: Data Service Operator DSS: Data Service Subscriber 38 Department of Electrical and Computer Engineering Wireless Network Virtualization Service

Layer Service Layer Service Layer Service Layer Service Providers Resource Layer Resource Layer Resource Layer Resource Layer Resource Providers

Infrastructure Layer Infrastructure Layer Infrastructure Layer Infrastructure Layer Infrastructure Providers Mobile Users Mobile Users Mobile Users Mobile Users 39 Department of Electrical and

Computer Engineering Utility Functions Unit price of receiving service from DSO i Workload rate Cost due to the delay DSS Indicating if DSS j is served by DSO i or not Amount of Computing Resource Blocks (CRBs) need to pay to DSO Cost of network delay Cost of computation delay where Service rate 40 Department of Electrical and

Computer Engineering Utility Functions Motivation price of FN k Amount of CRBs offloaded to FN k DSO Amount of CRBs served in massive data center servers Preferences of FN k to DSO i Amount of CRBs applied to server DSS j FN Increment cost if the FN k serves DSS j 41 Department of Electrical and Computer Engineering Problem Formulation DSS FN

DSO 42 Department of Electrical and Computer Engineering Hierarchical Game Framework Matching Game: DSOs and FNs Matching Games: FNs and DSSs Stackelberg Games: DSOs and DSSs 43 Department of Electrical and Computer Engineering Virtualized Network between DSOs and DSSs Second derivative less than zero

DSS First derivative equal to zero Put optimal amount of purchasing resource into optimization problem of DSO DSO 44 Matching between DSOs and FNs Department of Electrical and Computer Engineering Preference List DSO FN Algorithm DSO DSO ......

DSO DSO Motivation FN FN ...... FN FN 45 Matching between FNs and DSSs Department of Electrical and Computer Engineering Preference List FN DSS Algorithm

DSO Motivation Motivation ...... Motivation FN FN FN DSS DSS DSS Purchase Purchase ...... Purchase

DSO 46 Simulations Department of Electrical and Computer Engineering High Benefits and Necessity of FNs when the number of DSSs is large 47 Future Works Department of Electrical and Computer Engineering Introduction Hierarchical Game Framework A Complex Hierarchical Game Framework Future Works Conclusions Combining Hierarchical Game with

Reinforcement Learning 48 Complex Hierarchical Game Department of Electrical and Computer Engineering Service Layer Service Layer Service Layer Resource Layer Resource Layer Resource Layer Infrastructure Layer

Infrastructure Layer Infrastructure Layer Mobile Users Mobile Users Coalition Formation Position Switch Mobile Users 49 Multi-Agent Reinforcement Learning Department of Electrical and Computer Engineering + Learning rate

Discount factor ( , )=( ) ( , )+ [ ( , )+ ( ) ( ) ] ( ) ( )

Hierarchical Game Framework ( ) ( ) . . ( ) = ( )

Nash Equilibrium ( ) ( ) . . ( ) = ( ) Stackelberg Equilibrium 50 Department of Electrical and Computer Engineering Publications Book 1. H. Zhang, S. Khairy, L. X. Cai, and Z. Han, Resource Allocation in Unlicensed Long Term Evolution HetNets," contracting with Springer Science + Business Media, LLC. Book Chapter 2.

H. Zhang and Z. Han, Distributed Resource Allocation for Network Virtualization, Book chapter in Handbook of Cognitive Radio, Springer. Journal & Magazine 3. 4. 5. 6. 7. 8. 9. H. Zhang, Y. Zhang, Y. Gu, D. Niyato, and Z. Han, A Hierarchical Game Framework for Resource Management in Fog Computing, in IEEE Communication Magazine, vol. 55, no. 8, Aug. 2017. H. Zhang, Y. Xiao, S. Bu, D. Niyato, R. Yu, and Z. Han, Computing Resource Allocation in Three-Tier IoT Fog Networks: a Joint Optimization Approach Combining Stackelberg Game and Matching, in IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1204-1215, Oct. 2017. H. Zhang, Y. Xiao, L. X. Cai, D. Niyato, L. Song, and Z. Han, "A Multi-Leader Multi-Follower Stackelberg Game for Resource Management in LTE Unlicensed," in IEEE Transactions on Wireless Communications, vol. 16, no. 1, pp. 348-361, Jan. 2017. H. Zhang, D. Niyato, L. Song, T. Jiang, and Z. Han, Zero-determinant Strategy for Resource Sharing in Wireless Cooperation, IEEE Transactions on Wireless Communications, vol. 15, no. 3, pp. 21792192, Mar. 2016. H. Zhang, W. Ding, F. Yang, J. Song, and Z. Han, "Resource Allocation in Heterogeneous Network with Visible Light Communication and D2D: A Hierarchical Game Approach," submitted to Transactions on Communication. H. Zhang, B. Di, Z. Chang, X. Liu, L. Song, and Z. Han, "Equilibrium Problems with Equilibrium Constraints Analysis for Power Control and User Scheduling in NOMA Networks," submitted to Transactions on Wireless Communication. Z. Chang, L. Lei, H. Zhang, T. Ristaniemi, S. Chatzinotas, B. Ottersten, Z. Han, "Secure and Energy-Efficient Resource Allocation for Multiple-Antenna NOMA with Wireless Power Transfer," to be submitted to Transactions on Green Communications and Networking. Conference 10. H. Zhang, X. Tang, R. A. Banez, P. Ren, L. Song, and Z. Han, "An EPEC Analysis for Power Allocation in LTE-V Networks,"accepted for IEEE GLOBECOM 2017. 11. N. Raveendran, H. Zhang, Z. Zheng, L. Song, Z. Han, "Large-Scale Fog Computing Optimization using Equilibrium Problem with Equilibrium Constraints"accepted for IEEE GLOBECOM 2017. 12. X. Chen, H. Zhang, and Z. Han, "Delay-Tolerant Resource Scheduling in Large-Scale Virtualized Radio Access Networks," IEEE International Conference on Communications (ICC), Paris, France, May

2017. 13. H. Zhang, W. Ding, J. Song, and Z. Han, "A Hierarchical Game Approach for Visible Light Communication and Multi-Hop D2D Heterogeneous Network", 2016 IEEE Global Communications Conference (GLOBECOM), Washington, DC, Dec. 2016. 14. H. Zhang, X. Chen, and Z. Han, "A Zero-Determinant Approach for Power Control of Multiple Wireless Operators in LTE Unlicensed", 2016 IEEE Global Communications Conference (GLOBECOM), Washington, DC, Dec. 2016, 15. H. Zhang, Y. Xiao, S. Bu, D. Niyato, R. Yu, and Z. Han, Fog Computing in Multi-Tier Data Center Networks: A Hierarchical Game Approach, IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia, May 2016. 16. H. Zhang, Y. Xiao, L. X. Cai, L. Song, N. Dusit, and Z. Han, "A Hierarchical Game Approach for Multi-Operator Spectrum Sharing in LTE Unlicensed," IEEE Global Communications Conference, San Diego, CA, Dec. 2015. 17. H. Zhang, D. Niyato, L. Song, T. Jiang, and Z. Han, "Equilibrium Analysis for Zero-Determinant Strategy in Resource Management of Wireless Network," IEEE Wireless Communications and Networking Conference, New Orleans, LA, Mar. 2015. 18. H. Zhang, D. Niyato, L. Song, T. Jiang, and Z. Han, "Zero-Determinant Strategy in Cheating Management of Wireless Cooperation," IEEE Global Communications Conference, Austin, TX, Dec. 2014. 19. H. Zhang, M. Bennis, Luiz A. DaSilva, and Z. Han, "Multi-leader Multi-follower Stackelberg Game among Wi-Fi, Small Cell and Macrocell Networks," IEEE Global Communications Conference, Austin, TX, Dec. 2014. 20. H. Zhang, F. Li, N. Dusit, L. Song, T. Jiang, and Z. Han, "Zero-determinant Strategy in Power Control of Small Cell Network", invited, IEEE International Conference on Communication Systems (ICCS), Macau, China, Nov. 2014. 51 Department of Electrical and Computer Engineering Introduction Hierarchical Game Framework Conclusions

A Hierarchical Game Framework for Future Heterogeneous Network Architecture Hierarchical Game Analysis is Adopted For the Distributive Resource Allocation Multi-Tier Wireless Network Future Works LTE Unlicensed Fog Computing Zero-Determinant Strategy A Multi-Leader Multi-Follower Stackelberg Game A Three-Layer Hierarchical Game Conclusions Future Networks - Complex hierarchical game framework - A hierarchical game in reinforcement learning 52 Department of Electrical and Computer Engineering Thank You! Huaqing Zhang

[email protected] Wireless Networking, Signal Processing and Security Lab Department of Electrical and Computer Engineering University of Houston, TX, USA

Recently Viewed Presentations

  • 7 DI Model for Substance Use Prevention - Job Corps

    7 DI Model for Substance Use Prevention - Job Corps

    7D Addiction Treatment Model. Develops insight and awareness. Enhances positive movement through the stages of change. Initiates health and wellness. Manages stress . Prevents physical and mental disorders, Builds resiliency. Measures recovery progress to improve overall wellness
  • Teaching and Learning Research Programme The Recent Development

    Teaching and Learning Research Programme The Recent Development

    The Recent Development of Educational Research Capacity The case of the Teaching and Learning Research Program (TLRP) Andrew Pollard, Director, TLRP
  • Pre-exposure Clinical Trauma Training for Aeromedical ...

    Pre-exposure Clinical Trauma Training for Aeromedical ...

    The practice change is utilizing the newly appointed APN/FN disaster specialist to design and manage a pre-exposure clinical training in AE squadrons by developing specific types of trauma/injuries seen in current operations as means to acclimate all FN/AET to the...
  • PowerPoint Presentation

    PowerPoint Presentation

    Arial Times New Roman Wingdings Symbol Warstwy Microsoft Equation 3.0 Obraz - mapa bitowa In search of K+K- molecule Slajd 2 Physical motivation Kaon pair production in nucleons collisions Kaon pair production in nucleons collisions Cooler Synchrotron COSY COSY-11 apparatus...
  • FEDERAL WORK STUDY WCU FINANCIAL AID WHAT IS

    FEDERAL WORK STUDY WCU FINANCIAL AID WHAT IS

    Flexible work schedule. Chance to gain work experience and build references for future employment. Opportunity to work in a field related to the student's major. FEDERAL WORK STUDY - ADVANTAGES TO STUDENT . FWS is available to undergraduate students.
  • PowerPoint Presentation

    PowerPoint Presentation

    General Guidelines. Input. Output. Steps in between. Not Language Specific. Only you need to understand it. Should not work. Steps: You must first define input (what I'm going to start with) and output (what comes out at the end of...
  • PowerPoint Presentation

    PowerPoint Presentation

    Hatchery exotic-origin rainbow trout stocks - South Tacoma Hatchery, Spokane Hatchery, Goldendale Hatchery, & Eell Springs Hatchery - all are California-origin stocks Resident O. mykiss sampled in lower Cedar River in May 2003 for genetic analysis More sampled fish…
  • Welcome to the FSU Career Portfolio! Coming Spring

    Welcome to the FSU Career Portfolio! Coming Spring

    Tours within the Career Portfolio guide students in planning and building their portfolio. Examples throughout the Portfolio show how different students have described their skills and experiences. The Career Portfolio is closely connected to the FSU Career Center, where students...