Suggestions for InnoDB files

Suggestions for InnoDB files

am 16.03.2011 06:37:47 von Adarsh Sharma

--------------030904040105090901040204
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable

Dear all,

I have doubt regarding the storage structure for Innodb files :

Our database server has the following paths :

/dev/sda5 69G 35G 32G 52% /hdd1-1
/dev/sdb1 274G 225G 36G 87% /hdd2-1
/dev/sdc5 274G 225G 36G 87% /hdd3-1
/dev/sdd5 274G 218G 43G 84% /hdd4-1
/dev/sde1 266G 184G 69G 73% /hdd5-1

Is it better to have innodb_file_per_table on. =20
or
innodb_data_file_path =
/hdd2-1/innodb_data1/ibdata1:8G;/hdd3-1/innodb_data1/ibdata2 :8G;/hdd4-1/i=
nnodb_data1/ibdata3:8G;/hdd2-1/innodb_data1/ibdata4:8G;/hdd3 -1/innodb_dat=
a1/ibdata5:8G;/hdd4-1/innodb_data1/ibdata6:8G;/hdd2-1/innodb _data1/ibdata=
7:8G;/hdd3-1/innodb_data1/ibdata8:8G;/hdd4-1/innodb_data1/ib data9:8G;/hdd=
2-1/innodb_data1/ibdata10:8G;/hdd3-1/innodb_data1/ibdata11:8 G;/hdd4-1/inn=
odb_data1/ibdata12:8G;/hdd2-1/innodb_data2/ibdata13:8G;/hdd3 -1/innodb_dat=
a2/ibdata14:8G;/hdd4-1/innodb_data2/ibdata15:8G;/hdd2-1/inno db_data2/ibda=
ta16:8G;/hdd3-1/innodb_data2/ibdata17:8G;/hdd4-1/innodb_data 2/ibdata18:8G=
;/hdd2-1/innodb_data2/ibdata19:8G;/hdd3-1/innodb_data2/ibdat a20:8G;/hdd4-=
1/innodb_data2/ibdata21:8G;/hdd2-1/innodb_data2/ibdata22:8G; /hdd3-1/innod=
b_data2/ibdata23:8G;/hdd4-1/innodb_data2/ibdata24:8G;/hdd2-1 /innodb_data3=
/ibdata25:8G;/hdd3-1/innodb_data3/ibdata26:8G;/hdd4-1/innodb _data3/ibdata=
27:8G;/hdd2-1/innodb_data3/ibdata28:8G;/hdd3-1/innodb_data3/ ibdata29:8G;/=
hdd4-1/innodb_data3/ibdata30:8G;/hdd2-1/innodb_data3/ibdata3 1:8G;/hdd3-1/=
innodb_data3/ibdata32:8G;/hdd4-1/innodb_data3/ibdata33:8G;/h dd2-1/innodb_=
data3/ibdata34:8G;/hdd3-1/innodb_data3/ibdata35:8G;/hdd4-1/i nnodb_data3/i=
bdata36:8G;/hdd2-1/innodb_data4/ibdata37:8G;/hdd3-1/innodb_d ata4/ibdata38=
:8G;/hdd4-1/innodb_data4/ibdata39:8G;/hdd2-1/innodb_data4/ib data40:8G;/hd=
d3-1/innodb_data4/ibdata41:8G;/hdd4-1/innodb_data4/ibdata42: 8G;/hdd2-1/in=
nodb_data4/ibdata43:8G;/hdd3-1/innodb_data4/ibdata44:8G;/hdd 4-1/innodb_da=
ta4/ibdata45:8G;/hdd2-1/innodb_data4/ibdata46:8G;/hdd3-1/inn odb_data4/ibd=
ata47:8G;/hdd4-1/innodb_data4/ibdata48:8G;/hdd2-1/innodb_dat a5/ibdata49:8=
G;/hdd3-1/innodb_data5/ibdata50:8G;/hdd4-1/innodb_data5/ibda ta51:8G;/hdd2=
-1/innodb_data5/ibdata52:8G;/hdd3-1/innodb_data5/ibdata53:8G ;/hdd4-1/inno=
db_data5/ibdata54:8G;/hdd2-1/innodb_data5/ibdata55:8G;/hdd3- 1/innodb_data=
5/ibdata56:8G;/hdd4-1/innodb_data5/ibdata57:8G;/hdd2-1/innod b_data5/ibdat=
a58:8G;/hdd3-1/innodb_data5/ibdata59:8G;/hdd4-1/innodb_data5 /ibdata60:8G;=
/hdd2-1/innodb_data6/ibdata61:8G;/hdd3-1/innodb_data6/ibdata 62:8G;/hdd2-1=
/innodb_data6/ibdata63:8G;/hdd3-1/innodb_data6/ibdata64:8G;/ hdd2-1/innodb=
_data6/ibdata65:8G;/hdd3-1/innodb_data6/ibdata66:8G;/hdd2-1/ innodb_data6/=
ibdata67:8G;/hdd3-1/innodb_data6/ibdata68:8G;/hdd2-1/innodb_ data7/ibdata6=
9:8G;/hdd3-1/innodb_data7/ibdata70:8G;/hdd2-1/innodb_data7/i bdata71:8G;/h=
dd3-1/innodb_data7/ibdata72:8G;/hdd2-1/innodb_data7/ibdata73 :8G;/hdd3-1/i=
nnodb_data7/ibdata74:8G;/hdd2-1/innodb_data7/ibdata75:8G;/hd d3-1/innodb_d=
ata7/ibdata76:8G;/hdd4-1/innodb_data6/ibdata77:8G;/hdd4-1/in nodb_data6/ib=
data78:8G;/hdd4-1/innodb_data6/ibdata79:8G;/hdd4-1/innodb_da ta6/ibdata80:=
8G;/hdd4-1/innodb_data7/ibdata81:8G;/hdd5-1/innodb_data1/ibd ata82:8G;/hdd=
5-1/innodb_data1/ibdata83:8G;/hdd5-1/innodb_data1/ibdata84:8 G;/hdd5-1/inn=
odb_data1/ibdata85:8G;/hdd5-1/innodb_data2/ibdata86:8G;/hdd5 -1/innodb_dat=
a2/ibdata87:8G;/hdd5-1/innodb_data2/ibdata88:8G;/hdd5-1/inno db_data2/ibda=
ta89:8G;/hdd5-1/innodb_data3/ibdata90:8G;/hdd5-1/innodb_data 3/ibdata91:8G=
;/hdd5-1/innodb_data3/ibdata92:8G;/hdd5-1/innodb_data3/ibdat a93:8G;/hdd5-=
1/innodb_data4/ibdata94:8G;/hdd5-1/innodb_data4/ibdata95:8G; /hdd5-1/innod=
b_data4/ibdata96:8G;/hdd5-1/innodb_data4/ibdata97:8G;/hdd5-1 /innodb_data5=
/ibdata98:8G;/hdd5-1/innodb_data5/ibdata99:8G;/hdd5-1/innodb _data5/ibdata=
100:8G;/hdd5-1/innodb_data5/ibdata101:8G:autoextend

which is currently set because to increase performance to read from=20
separate small files instead of reading from one large one because one=20
table is expected to grow more than 300 GB & some tables are near about=20
60-80 GB & increasing day by day.

or

Make sure the disk /hdd2-1/innodb_data1 is big enough
/hdd2-1/innodb_data1 is going to need be a large RAID10 set

=20


MyISAM tables get stored in one /hdd1-1 directory.
Tables size grows day by day . Infact, one InnoDBtable is increasing=20
3GB/per day .

What is the best configuration for them so that we doesn't hit=20
performance issues.



Thanks & best Regards,

Adarsh Sharma



--------------030904040105090901040204--

Re: Suggestions for InnoDB files

am 16.03.2011 07:58:53 von Johan De Meersman

----- Original Message -----
> From: "Adarsh Sharma"
>
> Dear all,
>
> I have doubt regarding the storage structure for Innodb files :
>
> Our database server has the following paths :
>
> /dev/sda5 69G 35G 32G 52% /hdd1-1
> /dev/sdb1 274G 225G 36G 87% /hdd2-1
> /dev/sdc5 274G 225G 36G 87% /hdd3-1
> /dev/sdd5 274G 218G 43G 84% /hdd4-1
> /dev/sde1 266G 184G 69G 73% /hdd5-1

Interesting, but why like this instead of simply larger disks or raidsets ?

> Is it better to have innodb_file_per_table on.
> or
> innodb_data_file_path =
> /hdd2-1/innodb_data1/ibdata1:8G;/hdd3-1/innodb_data1/ibdata2 :8G;/hdd4-1/innodb_data1/ibdata3:8G;/hdd2-1/innodb_data1/ibd ata4:8G;
[unmanageable mess cut]

Why would you use 8G datafiles instead of large, partition-filling ones?

> which is currently set because to increase performance to read from
> separate small files instead of reading from one large one because
> one table is expected to grow more than 300 GB & some tables are near
> about 60-80 GB & increasing day by day.

I should check up on InnoDB internals wether it strips across datafiles, but from a disk point of view, many smaller files aren't likely to be faster than one large one.

> Make sure the disk /hdd2-1/innodb_data1 is big enough
> /hdd2-1/innodb_data1 is going to need be a large RAID10 set

A good RAID10 is recommended for databases anyway; I suggest you go with that.

> What is the best configuration for them so that we doesn't hit
> performance issues.

Performance issues are oftentimes more dependant on how you use the DB than how you set it up; but a good setup never hurt anyone, of course.

Consider throwing all those disk partitions into a single RAID10 set, either through underlying hardware or using MD on Linux. Even if you already have hardware RAID under those devices and can't modify that, consider concatenating the individual devices with LVM to benefit from striping.

InnoDB file-per-table should yield roughly the same performance as global datafiles, albeit with more file descriptors used. If you want to be able to reclaim space, go for file-per-table; if all space is for InnoDB anyway, monolithic storage might be slightly more convenient. I seem to recall InnoDB can use raw devices, too; I'm not sure wether there's a big performance gain, though. I that Oracle has stepped down from recommending that in recent years, stating only marginal gains.


--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: Suggestions for InnoDB files

am 16.03.2011 08:42:26 von Johan De Meersman

> From: "Adarsh Sharma"
>
> Johan De Meersman wrote:
> > Interesting, but why like this instead of simply larger disks or raidse=
ts ?
>
> It's the IT-Admin Issue , I can't question that and we have only disks of=
300GB ( SAS ).

Your admin is supposed to provide services that benefit the application you=
need to run on the server. You're stuck with the hardware, but not the set=
up.


> > Why would you use 8G datafiles instead of large, partition-filling ones=
?
>
> What is your recommendations for number of ibdata files , keeping in Mind=
Raid10 is not used and the size of tables .
> Because in RAID10 :
>
> We can utilize 50 â€=93 55 percent size of hard disk.(50-55 % of 4 ha=
rd disk total space if hard disks are 500 GB X 4 then we can
> utilize only 1 TB space from 2 TB.

Correct. That's the price you pay for the performance and redundancy RAID10=
gives you. Nothing is free in life :-) Incidentally, it's going to be exac=
tly 50% - I'll be very interested to see where he pulls those extra 5% from=
..

You could ostensibly go for RAID5, which will allow you to use 1.5 TB off t=
hose same four disks, at a minor loss of disk redundancy (only one may fail=
) and some loss of performance - but still better than no RAID at all. If y=
ou want to lose no space at all, use RAID0 (striping) to increase performan=
ce, but that offers no disk redundancy at all - single disk fails, you lose=
all data.

As a small overview, RAID 10 gives you the benefits of striping (data for a=
single file is split over multiple disks) so reads and writes faster, AND =
of mirrorring (every block is available on multiple disks, which provides i=
nsurance data loss when a disk breaks and additionally increases the read s=
peed even more. You won't actually quadruple the read speed, but I wouldn't=
be surprised to see it triple on a 4-disk RAID 10.

RAID 5 uses one of your disks for redundancy purposes, so any single disk m=
ay fail and you'll still have all your data. Data is striped, so disk perfo=
rmance also increases, although not as much as mirrorring. This is however =
the most CPU-intensive form, as checksumming over all disks happens at ever=
y write. This also makes that write speed won't see as much benefit.

RAID 0 has no redundancy whatsoever - if anything you could say it's worse =
than data over multiple disks, because if one disk fails the entire volume =
is lost. Because it offers striping, however, it gives performance a good b=
oost.


> Software RAID is not reliable on production environment because software =
raid is dependent on hardware and software both thing
> if one thing go down then it will not work, but in hardware raid there is=
no role of software every thing is depend on hardware.
> But, We are not able to afford Hardware RAID.

Maybe you shouldn't have an OS then, either; because if that fails everythi=
ng is down? My word, if that's his excuse, I seriously recommend you get a =
better admin.

Software RAID offers the same or better performance than hardware RAID, sav=
e for the real high-end RAID cards. Additionally it offers more flexibility=
in the setup - many combinations of RAID levels are possible, whereas the =
majority of controllers offer 1, 5 and 10 at most.

An additional benefit that is not to be laughed at, especially if you're on=
a budget, is that software RAID will work regardless of the hardware invol=
ved. Hardware RAID controllers tend to have their own specific set of metad=
ata on the disks, and if your controller breaks, you had better manage to g=
et the exact same one, or you risk not being able to read your disks. Sofwa=
re RAID, by virtue of being software, can simply be reinstalled on another =
system if need be. Tell MD to scan for and assemble RAID arrays and it'll j=
ust find the appropriate partitions and match the pieces together. No more =
accidentally putting a disk in the wrong bay and having it break the RAIDse=
t. (I'll admit that has become rare with controllers getting smarter over t=
he years, but I've seen multi-terabyte arrays go useless because some idiot=
operator switched two disks into the wrong bays)


So, yes, my recommendation remains the same: switch the system to software =
RAID; preferably 10, 5 or 0 if you really need all that space.


--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

Re: Suggestions for InnoDB files

am 16.03.2011 10:33:07 von Adarsh Sharma

--------------060105050205010301020803
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Johan De Meersman wrote:
>> From: "Adarsh Sharma"
>>
>> Johan De Meersman wrote:
>>
>>> Interesting, but why like this instead of simply larger disks or raidsets ?
>>>
>> It's the IT-Admin Issue , I can't question that and we have only disks of 300GB ( SAS ).
>>
>
> Your admin is supposed to provide services that benefit the application you need to run on the server. You're stuck with the hardware, but not the setup.
>
>
>
>>> Why would you use 8G datafiles instead of large, partition-filling ones?
>>>
>> What is your recommendations for number of ibdata files , keeping in Mind Raid10 is not used and the size of tables .
>> Because in RAID10 :
>>
>> We can utilize 50 – 55 percent size of hard disk.(50-55 % of 4 hard disk total space if hard disks are 500 GB X 4 then we can
>> utilize only 1 TB space from 2 TB.
>>
>
> Correct. That's the price you pay for the performance and redundancy RAID10 gives you. Nothing is free in life :-) Incidentally, it's going to be exactly 50% - I'll be very interested to see where he pulls those extra 5% from.
>
> You could ostensibly go for RAID5, which will allow you to use 1.5 TB off those same four disks, at a minor loss of disk redundancy (only one may fail) and some loss of performance - but still better than no RAID at all. If you want to lose no space at all, use RAID0 (striping) to increase performance, but that offers no disk redundancy at all - single disk fails, you lose all data.
>
> As a small overview, RAID 10 gives you the benefits of striping (data for a single file is split over multiple disks) so reads and writes faster, AND of mirrorring (every block is available on multiple disks, which provides insurance data loss when a disk breaks and additionally increases the read speed even more. You won't actually quadruple the read speed, but I wouldn't be surprised to see it triple on a 4-disk RAID 10.
>
> RAID 5 uses one of your disks for redundancy purposes, so any single disk may fail and you'll still have all your data. Data is striped, so disk performance also increases, although not as much as mirrorring. This is however the most CPU-intensive form, as checksumming over all disks happens at every write. This also makes that write speed won't see as much benefit.
>
> RAID 0 has no redundancy whatsoever - if anything you could say it's worse than data over multiple disks, because if one disk fails the entire volume is lost. Because it offers striping, however, it gives performance a good boost.
>
>
>
>> Software RAID is not reliable on production environment because software raid is dependent on hardware and software both thing
>> if one thing go down then it will not work, but in hardware raid there is no role of software every thing is depend on hardware.
>> But, We are not able to afford Hardware RAID.
>>
>
> Maybe you shouldn't have an OS then, either; because if that fails everything is down? My word, if that's his excuse, I seriously recommend you get a better admin.
>
> Software RAID offers the same or better performance than hardware RAID, save for the real high-end RAID cards. Additionally it offers more flexibility in the setup - many combinations of RAID levels are possible, whereas the majority of controllers offer 1, 5 and 10 at most.
>
> An additional benefit that is not to be laughed at, especially if you're on a budget, is that software RAID will work regardless of the hardware involved. Hardware RAID controllers tend to have their own specific set of metadata on the disks, and if your controller breaks, you had better manage to get the exact same one, or you risk not being able to read your disks. Sofware RAID, by virtue of being software, can simply be reinstalled on another system if need be. Tell MD to scan for and assemble RAID arrays and it'll just find the appropriate partitions and match the pieces together. No more accidentally putting a disk in the wrong bay and having it break the RAIDset. (I'll admit that has become rare with controllers getting smarter over the years, but I've seen multi-terabyte arrays go
useless because some idiot operator switched two disks into the wrong bays)
>
>
> So, yes, my recommendation remains the same: switch the system to software RAID; preferably 10, 5 or 0 if you really need all that space.
>
>
>

A Heartiest Thanks from my heart for explaining all these things in a
fantastic manner. I agreed with your suggestions but one thing which
isn't explained from your side , as you go deeper in RAID point.

Q:- What is your recommendations for number of ibdata files , would it be

Make sure the disk /hdd2-1/innodb_data1 is big enough and it doesn't affect performance.


I need your help while configuring RAID10 on a Server, may be next week.


Best Regards,
Adarsh Sharma




--------------060105050205010301020803--

Re: Suggestions for InnoDB files

am 16.03.2011 10:39:47 von Johan De Meersman

--=_c5dc2ed0-d9f8-4a75-a590-22d1574cad8f
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

----- Original Message -----

> From: "Adarsh Sharma"

> Johan De Meersman wrote:
> A Heartiest Thanks from my heart for explaining all these things in a
> fantastic manner. I agreed with your suggestions but one thing which
> isn't explained from your side , as you go deeper in RAID point.

> Q:- What is your recommendations for number of ibdata files , would
> it be
> Make sure the disk /hdd2-1/innodb_data1 is big enough and it doesn't
> affect performance.
Roughly, yes - file-per-table is only useful if you need to be able to reclaim the space for non-InnoDB data; and I don't think InnoDB stripes across datafiles, so just use one large file on one RAID partition. Saves on file descriptors, saves on header space.

--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

--=_c5dc2ed0-d9f8-4a75-a590-22d1574cad8f--

RE: Suggestions for InnoDB files

am 16.03.2011 18:26:07 von Rolando Edwards

WW91IHNob3VsZCB1c2UgYSBzaW1wbCBkYXRhIHBhdGggYW5kIGNyZWF0ZSBh IHNlcGFyYXRlIHRh
Ymxlc3BhY2UgZm9yIGVhY2ggSW5ub0RCIGZpbGUNCg0KaW5ub2RiX2RhdGFf ZmlsZV9wYXRoPWli
ZGF0YTE6MTBNOmF1dG9leHRlbmQNCmlubm9kYl9maWxlX3Blcl90YWJsZQ0K DQpUaGlzIHdheSwg
aWJkYXRhMSBvbmx5IGNvbnRhaW5zIHRoZSBtZXRhZGF0YSBhbmQgTVZDQyBj b250cm9sIGRhdGEg
Zm9yIGFsbCBJbm5vREIgZmlsZXMgYW5kIHRyYW5zYWN0aW9ucw0KDQpBd2hp bGUgYmFjaywgeW91
IHJhbiBhIHF1ZXJ5IHRvIGdldCB0aGUgRGlza3NwYWNlIHVzZWQgZnJvbSB5 b3UgZGF0YSBhbmQg
c2VudCBiYWNrIHRoaXM6DQoNCistLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0t LS0tLS0tLS0tLS0t
LS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0t LS0tKw0KfCBTdG9y
YWdlIEVuZ2luZSB8IERhdGEgU2l6ZSAgICAgICAgICAgIHwgSW5kZXggU2l6 ZSAgICAgICAgICAg
fCBUYWJsZSBTaXplICAgICAgICAgICB8DQorLS0tLS0tLS0tLS0tLS0tLSst LS0tLS0tLS0tLS0t
LS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0tLS0tLS0t LS0tLS0tLS0tLSsN
CnwgTXlJU0FNICAgICAgICAgfCAgICAgICAgICAgICAwLjAxMCBUQiB8ICAg ICAgICAgICAgIDAu
MDAxIFRCIHwgICAgICAgICAgICAgMC4wMTEgVEIgfCANCnwgSW5ub0RCICAg ICAgICAgfCAgICAg
ICAgICAgICAwLjE2MSBUQiB8ICAgICAgICAgICAgIDAuMDEwIFRCIHwgICAg ICAgICAgICAgMC4x
NzEgVEIgfCANCnwgVG90YWwgICAgICAgICAgfCAgICAgICAgICAgICAwLjE3 MSBUQiB8ICAgICAg
ICAgICAgIDAuMDExIFRCIHwgICAgICAgICAgICAgMC4xODIgVEIgfCANCist LS0tLS0tLS0tLS0t
LS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0t
LS0tLS0tLS0tLS0tLS0tKw0KDQpUaGlzIG1lYW5zIHlvdSBvbmx5IGhhdmUg c29tZXRoaW5nIGxp
a2UgMjAwR0Igb2YgZGF0YS4NCg0KWW91ciBhcmNoaXRlY3R1cmUgaGFzIDEw MSBmaWxlcyB0aGF0
IGFyZSA4RyBlYWNoIHRvIGhvdXNlIGFsbCBJbm5vREIgZGF0YS4gWW91IGhh ZCBhIDU1M0dCIHRh
YmxlIHdoaWNoIG11c3QgYmUgc3ByZWFkIG91dCBvdmVyIGF0IGxlYXN0IDY5 IG9mIHRob3NlIDhH
IGZpbGVzLg0KDQpZb3Ugc2hvdWxkIGNvbnZlcnQgb3ZlciB0byA2IHggMzAw R0IgUkFJRDEwIHNl
dCB3aGljaCB3aWxsIGdpdmUgeW91IDgyNEdCIG9mIHNwYWNlIHRvIHN0YXJ0 Lg0KDQpSb2xhbmRv
IEEuIEVkd2FyZHMNCk15U1FMIERCQSAoU0NNREJBKQ0KDQoxNTUgQXZlbnVl IG9mIHRoZSBBbWVy
aWNhcywgRmlmdGggRmxvb3INCk5ldyBZb3JrLCBOWSAxMDAxMw0KMjEyLTYy NS01MzA3IChXb3Jr
KQ0KMjAxLTY2MC0zMjIxIChDZWxsKQ0KQUlNICYgU2t5cGUgOiBSb2xhbmRv TG9naWNXb3J4DQpy
ZWR3YXJkc0Bsb2dpY3dvcmtzLm5ldA0KaHR0cDovL3d3dy5saW5rZWRpbi5j b20vaW4vcm9sYW5k
b2Vkd2FyZHMNCg0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJv bTogQWRhcnNoIFNo
YXJtYSBbbWFpbHRvOmFkYXJzaC5zaGFybWFAb3JrYXNoLmNvbV0gDQpTZW50 OiBXZWRuZXNkYXks
IE1hcmNoIDE2LCAyMDExIDU6MzMgQU0NClRvOiBKb2hhbiBEZSBNZWVyc21h bg0KQ2M6IG15c3Fs
QGxpc3RzLm15c3FsLmNvbQ0KU3ViamVjdDogUmU6IFN1Z2dlc3Rpb25zIGZv ciBJbm5vREIgZmls
ZXMNCg0KSm9oYW4gRGUgTWVlcnNtYW4gd3JvdGU6DQo+PiBGcm9tOiAiQWRh cnNoIFNoYXJtYSIg
PGFkYXJzaC5zaGFybWFAb3JrYXNoLmNvbT4NCj4+DQo+PiBKb2hhbiBEZSBN ZWVyc21hbiB3cm90
ZToNCj4+ICAgICANCj4+PiBJbnRlcmVzdGluZywgYnV0IHdoeSBsaWtlIHRo aXMgaW5zdGVhZCBv
ZiBzaW1wbHkgbGFyZ2VyIGRpc2tzIG9yIHJhaWRzZXRzID8NCj4+PiAgICAg ICANCj4+IEl0J3Mg
dGhlIElULUFkbWluIElzc3VlICwgSSBjYW4ndCBxdWVzdGlvbiB0aGF0IGFu ZCB3ZSBoYXZlIG9u
bHkgZGlza3Mgb2YgMzAwR0IgKCBTQVMgKS4NCj4+ICAgICANCj4NCj4gWW91 ciBhZG1pbiBpcyBz
dXBwb3NlZCB0byBwcm92aWRlIHNlcnZpY2VzIHRoYXQgYmVuZWZpdCB0aGUg YXBwbGljYXRpb24g
eW91IG5lZWQgdG8gcnVuIG9uIHRoZSBzZXJ2ZXIuIFlvdSdyZSBzdHVjayB3 aXRoIHRoZSBoYXJk
d2FyZSwgYnV0IG5vdCB0aGUgc2V0dXAuDQo+DQo+DQo+ICAgDQo+Pj4gV2h5 IHdvdWxkIHlvdSB1
c2UgOEcgZGF0YWZpbGVzIGluc3RlYWQgb2YgbGFyZ2UsIHBhcnRpdGlvbi1m aWxsaW5nIG9uZXM/
DQo+Pj4gICAgICAgDQo+PiBXaGF0IGlzIHlvdXIgcmVjb21tZW5kYXRpb25z IGZvciBudW1iZXIg
b2YgaWJkYXRhIGZpbGVzICwga2VlcGluZyBpbiBNaW5kIFJhaWQxMCBpcyBu b3QgdXNlZCBhbmQg
dGhlIHNpemUgb2YgdGFibGVzIC4NCj4+IEJlY2F1c2UgaW4gUkFJRDEwIDoN Cj4+DQo+PiBXZSBj
YW4gdXRpbGl6ZSA1MCDigJMgNTUgcGVyY2VudCBzaXplIG9mIGhhcmQgZGlz ay4oNTAtNTUgJSBv
ZiA0IGhhcmQgZGlzayB0b3RhbCBzcGFjZSBpZiBoYXJkIGRpc2tzIGFyZSA1 MDAgR0IgWCA0IHRo
ZW4gd2UgY2FuIA0KPj4gdXRpbGl6ZSBvbmx5IDEgVEIgc3BhY2UgZnJvbSAy IFRCLg0KPj4gICAg
IA0KPg0KPiBDb3JyZWN0LiBUaGF0J3MgdGhlIHByaWNlIHlvdSBwYXkgZm9y IHRoZSBwZXJmb3Jt
YW5jZSBhbmQgcmVkdW5kYW5jeSBSQUlEMTAgZ2l2ZXMgeW91LiBOb3RoaW5n IGlzIGZyZWUgaW4g
bGlmZSA6LSkgSW5jaWRlbnRhbGx5LCBpdCdzIGdvaW5nIHRvIGJlIGV4YWN0 bHkgNTAlIC0gSSds
bCBiZSB2ZXJ5IGludGVyZXN0ZWQgdG8gc2VlIHdoZXJlIGhlIHB1bGxzIHRo b3NlIGV4dHJhIDUl
IGZyb20uDQo+DQo+IFlvdSBjb3VsZCBvc3RlbnNpYmx5IGdvIGZvciBSQUlE NSwgd2hpY2ggd2ls
bCBhbGxvdyB5b3UgdG8gdXNlIDEuNSBUQiBvZmYgdGhvc2Ugc2FtZSBmb3Vy IGRpc2tzLCBhdCBh
IG1pbm9yIGxvc3Mgb2YgZGlzayByZWR1bmRhbmN5IChvbmx5IG9uZSBtYXkg ZmFpbCkgYW5kIHNv
bWUgbG9zcyBvZiBwZXJmb3JtYW5jZSAtIGJ1dCBzdGlsbCBiZXR0ZXIgdGhh biBubyBSQUlEIGF0
IGFsbC4gSWYgeW91IHdhbnQgdG8gbG9zZSBubyBzcGFjZSBhdCBhbGwsIHVz ZSBSQUlEMCAoc3Ry
aXBpbmcpIHRvIGluY3JlYXNlIHBlcmZvcm1hbmNlLCBidXQgdGhhdCBvZmZl cnMgbm8gZGlzayBy
ZWR1bmRhbmN5IGF0IGFsbCAtIHNpbmdsZSBkaXNrIGZhaWxzLCB5b3UgbG9z ZSBhbGwgZGF0YS4N
Cj4NCj4gQXMgYSBzbWFsbCBvdmVydmlldywgUkFJRCAxMCBnaXZlcyB5b3Ug dGhlIGJlbmVmaXRz
IG9mIHN0cmlwaW5nIChkYXRhIGZvciBhIHNpbmdsZSBmaWxlIGlzIHNwbGl0 IG92ZXIgbXVsdGlw
bGUgZGlza3MpIHNvIHJlYWRzIGFuZCB3cml0ZXMgZmFzdGVyLCBBTkQgb2Yg bWlycm9ycmluZyAo
ZXZlcnkgYmxvY2sgaXMgYXZhaWxhYmxlIG9uIG11bHRpcGxlIGRpc2tzLCB3 aGljaCBwcm92aWRl
cyBpbnN1cmFuY2UgZGF0YSBsb3NzIHdoZW4gYSBkaXNrIGJyZWFrcyBhbmQg YWRkaXRpb25hbGx5
IGluY3JlYXNlcyB0aGUgcmVhZCBzcGVlZCBldmVuIG1vcmUuIFlvdSB3b24n dCBhY3R1YWxseSBx
dWFkcnVwbGUgdGhlIHJlYWQgc3BlZWQsIGJ1dCBJIHdvdWxkbid0IGJlIHN1 cnByaXNlZCB0byBz
ZWUgaXQgdHJpcGxlIG9uIGEgNC1kaXNrIFJBSUQgMTAuDQo+DQo+IFJBSUQg NSB1c2VzIG9uZSBv
ZiB5b3VyIGRpc2tzIGZvciByZWR1bmRhbmN5IHB1cnBvc2VzLCBzbyBhbnkg c2luZ2xlIGRpc2sg
bWF5IGZhaWwgYW5kIHlvdSdsbCBzdGlsbCBoYXZlIGFsbCB5b3VyIGRhdGEu IERhdGEgaXMgc3Ry
aXBlZCwgc28gZGlzayBwZXJmb3JtYW5jZSBhbHNvIGluY3JlYXNlcywgYWx0 aG91Z2ggbm90IGFz
IG11Y2ggYXMgbWlycm9ycmluZy4gVGhpcyBpcyBob3dldmVyIHRoZSBtb3N0 IENQVS1pbnRlbnNp
dmUgZm9ybSwgYXMgY2hlY2tzdW1taW5nIG92ZXIgYWxsIGRpc2tzIGhhcHBl bnMgYXQgZXZlcnkg
d3JpdGUuIFRoaXMgYWxzbyBtYWtlcyB0aGF0IHdyaXRlIHNwZWVkIHdvbid0 IHNlZSBhcyBtdWNo
IGJlbmVmaXQuDQo+DQo+IFJBSUQgMCBoYXMgbm8gcmVkdW5kYW5jeSB3aGF0 c29ldmVyIC0gaWYg
YW55dGhpbmcgeW91IGNvdWxkIHNheSBpdCdzIHdvcnNlIHRoYW4gZGF0YSBv dmVyIG11bHRpcGxl
IGRpc2tzLCBiZWNhdXNlIGlmIG9uZSBkaXNrIGZhaWxzIHRoZSBlbnRpcmUg dm9sdW1lIGlzIGxv
c3QuIEJlY2F1c2UgaXQgb2ZmZXJzIHN0cmlwaW5nLCBob3dldmVyLCBpdCBn aXZlcyBwZXJmb3Jt
YW5jZSBhIGdvb2QgYm9vc3QuDQo+DQo+DQo+ICAgDQo+PiBTb2Z0d2FyZSBS QUlEIGlzIG5vdCBy
ZWxpYWJsZSBvbiBwcm9kdWN0aW9uIGVudmlyb25tZW50IGJlY2F1c2Ugc29m dHdhcmUgcmFpZCBp
cyBkZXBlbmRlbnQgb24gaGFyZHdhcmUgYW5kIHNvZnR3YXJlIGJvdGggdGhp bmcgDQo+PiBpZiBv
bmUgdGhpbmcgZ28gZG93biB0aGVuIGl0IHdpbGwgbm90IHdvcmssIGJ1dCBp biBoYXJkd2FyZSBy
YWlkIHRoZXJlIGlzIG5vIHJvbGUgb2Ygc29mdHdhcmUgZXZlcnkgdGhpbmcg aXMgZGVwZW5kIG9u
IGhhcmR3YXJlLg0KPj4gQnV0LCBXZSBhcmUgbm90IGFibGUgdG8gYWZmb3Jk IEhhcmR3YXJlIFJB
SUQuDQo+PiAgICAgDQo+DQo+IE1heWJlIHlvdSBzaG91bGRuJ3QgaGF2ZSBh biBPUyB0aGVuLCBl
aXRoZXI7IGJlY2F1c2UgaWYgdGhhdCBmYWlscyBldmVyeXRoaW5nIGlzIGRv d24/IE15IHdvcmQs
IGlmIHRoYXQncyBoaXMgZXhjdXNlLCBJIHNlcmlvdXNseSByZWNvbW1lbmQg eW91IGdldCBhIGJl
dHRlciBhZG1pbi4NCj4NCj4gU29mdHdhcmUgUkFJRCBvZmZlcnMgdGhlIHNh bWUgb3IgYmV0dGVy
IHBlcmZvcm1hbmNlIHRoYW4gaGFyZHdhcmUgUkFJRCwgc2F2ZSBmb3IgdGhl IHJlYWwgaGlnaC1l
bmQgUkFJRCBjYXJkcy4gQWRkaXRpb25hbGx5IGl0IG9mZmVycyBtb3JlIGZs ZXhpYmlsaXR5IGlu
IHRoZSBzZXR1cCAtIG1hbnkgY29tYmluYXRpb25zIG9mIFJBSUQgbGV2ZWxz IGFyZSBwb3NzaWJs
ZSwgd2hlcmVhcyB0aGUgbWFqb3JpdHkgb2YgY29udHJvbGxlcnMgb2ZmZXIg MSwgNSBhbmQgMTAg
YXQgbW9zdC4NCj4NCj4gQW4gYWRkaXRpb25hbCBiZW5lZml0IHRoYXQgaXMg bm90IHRvIGJlIGxh
dWdoZWQgYXQsIGVzcGVjaWFsbHkgaWYgeW91J3JlIG9uIGEgYnVkZ2V0LCBp cyB0aGF0IHNvZnR3
YXJlIFJBSUQgd2lsbCB3b3JrIHJlZ2FyZGxlc3Mgb2YgdGhlIGhhcmR3YXJl IGludm9sdmVkLiBI
YXJkd2FyZSBSQUlEIGNvbnRyb2xsZXJzIHRlbmQgdG8gaGF2ZSB0aGVpciBv d24gc3BlY2lmaWMg
c2V0IG9mIG1ldGFkYXRhIG9uIHRoZSBkaXNrcywgYW5kIGlmIHlvdXIgY29u dHJvbGxlciBicmVh
a3MsIHlvdSBoYWQgYmV0dGVyIG1hbmFnZSB0byBnZXQgdGhlIGV4YWN0IHNh bWUgb25lLCBvciB5
b3UgcmlzayBub3QgYmVpbmcgYWJsZSB0byByZWFkIHlvdXIgZGlza3MuIFNv ZndhcmUgUkFJRCwg
YnkgdmlydHVlIG9mIGJlaW5nIHNvZnR3YXJlLCBjYW4gc2ltcGx5IGJlIHJl aW5zdGFsbGVkIG9u
IGFub3RoZXIgc3lzdGVtIGlmIG5lZWQgYmUuIFRlbGwgTUQgdG8gc2NhbiBm b3IgYW5kIGFzc2Vt
YmxlIFJBSUQgYXJyYXlzIGFuZCBpdCdsbCBqdXN0IGZpbmQgdGhlIGFwcHJv cHJpYXRlIHBhcnRp
dGlvbnMgYW5kIG1hdGNoIHRoZSBwaWVjZXMgdG9nZXRoZXIuIE5vIG1vcmUg YWNjaWRlbnRhbGx5
IHB1dHRpbmcgYSBkaXNrIGluIHRoZSB3cm9uZyBiYXkgYW5kIGhhdmluZyBp dCBicmVhayB0aGUg
UkFJRHNldC4gKEknbGwgYWRtaXQgdGhhdCBoYXMgYmVjb21lIHJhcmUgd2l0 aCBjb250cm9sbGVy
cyBnZXR0aW5nIHNtYXJ0ZXIgb3ZlciB0aGUgeWVhcnMsIGJ1dCBJJ3ZlIHNl ZW4gbXVsdGktdGVy
YWJ5dGUgYXJyYXlzIGdvIHVzZWxlc3MgYmVjYXVzZSBzb21lIGlkaW90IG9w ZXJhdG9yIHN3aXRj
aGVkIHR3byBkaXNrcyBpbnRvIHRoZSB3cm9uZyBiYXlzKQ0KPg0KPg0KPiBT bywgeWVzLCBteSBy
ZWNvbW1lbmRhdGlvbiByZW1haW5zIHRoZSBzYW1lOiBzd2l0Y2ggdGhlIHN5 c3RlbSB0byBzb2Z0
d2FyZSBSQUlEOyBwcmVmZXJhYmx5IDEwLCA1IG9yIDAgaWYgeW91IHJlYWxs eSBuZWVkIGFsbCB0
aGF0IHNwYWNlLg0KPg0KPg0KPiAgIA0KDQpBIEhlYXJ0aWVzdCBUaGFua3Mg ZnJvbSBteSBoZWFy
dCBmb3IgZXhwbGFpbmluZyBhbGwgdGhlc2UgdGhpbmdzIGluIGEgDQpmYW50 YXN0aWMgbWFubmVy
LiBJIGFncmVlZCB3aXRoIHlvdXIgc3VnZ2VzdGlvbnMgYnV0IG9uZSB0aGlu ZyB3aGljaCANCmlz
bid0IGV4cGxhaW5lZCBmcm9tIHlvdXIgc2lkZSAsIGFzIHlvdSBnbyBkZWVw ZXIgaW4gUkFJRCBw
b2ludC4NCg0KUTotIFdoYXQgaXMgeW91ciByZWNvbW1lbmRhdGlvbnMgZm9y IG51bWJlciBvZiBp
YmRhdGEgZmlsZXMgLCB3b3VsZCBpdCBiZQ0KDQpNYWtlIHN1cmUgdGhlIGRp c2sgL2hkZDItMS9p
bm5vZGJfZGF0YTEgaXMgYmlnIGVub3VnaCBhbmQgaXQgZG9lc24ndCBhZmZl Y3QgcGVyZm9ybWFu
Y2UuDQoNCg0KSSBuZWVkIHlvdXIgaGVscCB3aGlsZSBjb25maWd1cmluZyBS QUlEMTAgb24gYSBT
ZXJ2ZXIsIG1heSBiZSBuZXh0IHdlZWsuDQoNCg0KQmVzdCBSZWdhcmRzLA0K QWRhcnNoIFNoYXJt
YQ0KDQoNCg0K