发信人: zhaoce (米高蜥蜴), 信区: Military
标 题: 天朝现在风起云涌的去ioe运动
发信站: BBS 未名空间站 (Wed May 28 05:41:48 2014, 美东)
谈谈去 IOE 运动
何谓 IOE ?
所谓 IOE 是个简称。是指以 IBM 、Oracle、EMC 为代表的小型机、集中式数据库和高
端存储的技术架构。其中 I 指 IBM p 系列小型机，操作系统是 AIX，IBM 专有的
Unix 系统；O 指 Oracle 数据库(RDBMS)；E 指 EMC 中高端 SAN 存储，曾经一度是
IT 企业很喜欢采用的技术架构。IOE 这个说法怎么来的? 据我所知应是来自阿里技术
在小型机领域，只有 IBM 这一家，独步武林；HP 当初把宝押在安腾上，算是早早退出
这个市场；Sun 日薄西山，SPARC 机器…那就更不必说了。另外，需要说明的是，IBM
也生产存储产品，但 IBM 的存储产品早期其实挺山寨，竞争不过 EMC ，而且有些用户
会忌讳把所有的东西困在一家公司身上，尾大不掉。 起码在国内，EMC 的占有率应该
软件方面，在当初几乎没的选择，只有 Oracle 这一家，IBM 的 DB2 实在是不行，虽
然号称市场占有率不错。国内用 Oracle 数据库支撑互联网应用的话，一般是采用
Data Guard 这个架构方案。
或许有人认为「去 IOE」会节约企业成本，实际上，当时的 Oracle 和 EMC 等软件成
从另外一个角度考虑，尤其从运维 DBA 的角度去审视，「去 IOE」 实际上是必须要进
行的，或者说去「O」是必须的，因为当时存在的问题是，Oracle 数据库对用户 (DBA)
来说已经不够灵活，常用的 Data Guard 模式无法适应互联网公司快速增长，最基本
向扩展。或许有人说，不是有 RAC 么? 但 Oracle RAC 是无法对付高并发下的 OLTP
应用的 – 一直到现在很多人都认识不到这一点，RAC 跑跑数据仓库什么的倒是不错。
注：有人会说 Orale RDBMS 11g 的 Data Guard 可以读写分离呀，这个所谓的读写分
Oracle RAC 怎么就不能应付 OLTP 呢? 别争论了，你非要说可以应付，没问题，但是
在阿里体系的公司里，还真没人敢这么玩儿，为什么? 是做不到? 还是他们掉进坑过?
如果要动「O」，那么 「I」 和「E」就必须要动 – 相信不会有人在小型机上跑
2.开源软件的快速成熟。举个例子，这两年 MySQL 体系的软件进步相当惊人，各种经
、淘宝)在使用 MySQL 的同时也将其技术改进回馈给技术社区，把技术方案分享给业界
到的。要知道物理机械硬盘存储的性能数十年几乎没得到什么大的改进 – 当然每年提
升一点是有的。但 SSD 相比机械硬盘来说，则是质的飞跃。我记忆深刻的是，每年做
I/O 容量规划的时候都会发愁，因为即使已经使用上了很高端的 EMC 存储设备，但实
际上只要应用层 I/O 没有命中到存储内存，直接打到后面的磁盘上，几乎没什么抵抗
能力。比如当时一个硬盘极限能撑 100 多个 I/O，100 块硬盘也不过是万把个 I/O 就
不行了。 但这样的 I/O 「打击」对 SSD 来说，则不是什么大问题。SSD 给解决「IOE
」体系最大的瓶颈 – I/O 能力提供了硬件先决条件。
提到摩尔定律，重点要说的 X86 芯片的计算能力不断进步，而 IBM 的 Power 芯片虽
然也在不断进步，但正式商用的节奏明显在控制。这就给 Intel 体系带来了机会和空
值得警惕的。去掉 Oracle 不意味着就要采用国产的垃圾数据库，因为 MySQL 以及衍
生的各种分支数据库才是最佳选择。同样，不用 IBM 的小型机也不意味着国产服务器
2014 以后会有更多公司「去 IOE」
从目前的种种趋势来看，在今后几年，国内一些互联网公司以及 IT 企业会逐渐的「去
至都不需要从头开始进行储备 – 类似「沃趣科技」这样的团队已经能够提供足够好的
。或许这也是 IOE 们被抛弃的一个原因，也可能是一些创业团队的新机会。
关于 IOPS 的数据补充：
机械硬盘现在最高号称能跑到 400 IOPS，但应该 200 左右(走 SAS 接口)也就是极限
了; 单块 SSD(走 PCIe 接口)的最高记录是九百多万，用不了多久突破千万 IOPS 是没
Oracle Distributed Document Capture 10gR3 install and basic run setup
By W.ZH Apr. 2013In Oracle Distributed Document Capture, users scan groups of pages in batches, using
settings saved by an administrator in a scan profile. A batch is divided into one or more
documents, and each document contains one or more pages
There is few doc article online give how to install the ODDC and how to start use it. This is just so poor product cared by somebody.
(ODC at this time is not available download from oracle website. maybe they are doing dev for a real 11g version.)
=============== How to install ODDC =========================================================
1. download the installer of ODDC
2. install ODDC on windows, before that you need to install the JAVA for it. please note that a 32bit java is need. then set the PATH for java bin folder in windows.
unzip and just run the setup to install it.
3. you need to read the install guide and do the basic configuration:
After install, what you can get is the several things:
1. A program called “Oracle Document Capture”
2. A program called “Distributed Document Capture Server Configuration”
3. A service in windows, registered as “Oracle Distributed Document Capture”
Here are things you need to config to make ODDC can be ready to work.
1. Go to “Oracle Document Capture”, to setup the Batch folder / Commit folder / Capture Database config
for those folders, you can create a main folder to contain them as there are still more folders needed later.
for the Database config, refer to install guide, simple way is to use the access DB first.
if you can not find the “Capture.mdb” under the <installed Dir>, then you can go to C;\ProgramData\Oracle\Document Capture\, copy it out and save it to <installed Dir>, then use this version.(do not point to that runtime version!)
2. Go to “Distributed Document Capture Server Configuration”,
set those folders, like, Error Batch location and Log file location.
The “Set Client Access Folders” is the folder named “ClientAccess” under the <install dir>\WebPages\ , this folder is going to be the location that client upload their PAK files to web server.
The others just follow the install the guide.
3. Configuring the IIS server,
You need to start a IIS server in your windows.
Enable the IIS manage console, and also setup the ASP and related filters, just follow the install guide to do these.
Then create the application pool and a website(or a virtual dir) in the IIS,
Deploy the <install dir>\WebPages folder to your new created website. we suppose the website can be http://10.10.10.10:888/*** as there is no index.html in the folder, just point to any file to make sure your site runs.
4. Give folder permission,
there is a user “IUSR” in the windows, work for IIS, you need give all the folders that we have setup in step 1,2,3, include <install dir> the read write permissions
5. let us make a verfiy,
Try to access admin URL of ODDC.
If you can login and see the admin UI (by Java applet?), then your IIS works. we call this UI as the “Scan Profile Admin UI” later.
To this step, you seems set all things done, but still not able to understand that how system work…. what a user unfriendly product!
=============== How to make the ODDC run with your UCM 11g ===================================================
1. Make your Webcenter Content (UCM) installed and runs. After that a basic ODDC example will need you to config “File Cabinent” + “Scan Profile” + “Index/Pick List (Optional)” , then Client can access to server side to run.
2. Go to “Oracle Document Capture”,– admin to config the “File Cabinent”,
under “Commit Profile”, you an set it to link to UCM server or just save the received file to a folder.
for UCM, you can also create index fields and map them to the UCM internal metadata at there settings.
you can also setup the pick-list or DB lookup for index at here. if you have index asked, then later when client try to scan/send file, it is asked in input/select the indexed data together with the doc.
3. After you have the “File Cabinent”, then you go to
“Scan Profile Admin UI”, to use this File Cabinent to create a scan profile. just follow the admin guide to do these.
4. Go the windows services, start the service of “Oracle Distributed Document Capture”
5. finally you can go to Client to do a real scan and send/upload.
find a 32bit IE browser, as the web page need to run Active X control. Open the client URL like this: http://10.10.10.10:888/WebCapture.asp
after login you can follow the user guide, select the scan profile and to do the scan, input index data and also the send, etc.
5. check out the batch folder, ClientAccess folder at the server side. File will upload to IIS as PAK file under the ClientAccess folder, then the capture service will unzip it and process it in the batch folder.
after that, as we have config the commit to UCM. you can go to UCM to search the new checked in files.
and in the UCM, check the metadata of your files, if you have set the field mapping, then your index data will be checked in with your file to the UCM too.
And also read the log in the log folder.
Issue and solution:
1. when I set up the UCM 11g as the commit server, i get error of creating the Capture-UCM Implementation Object”
Solution: your windows do not have java installed, install the java and also the set the PATH.refer to https://forums.oracle.com/forums/thread.jspa?threadID=1091622
2. After i setup every thing, i can go to client to import and send file, and it shows successful, but i can not find the file checked in the UCM side.
Solution: you should check the folder of the ClientAccess first, if you find the PAK file inside here, but the batch folder has nothing happen.
then reason should be you do not start the service in the windows. just do it and can fix this issue.
Please note that this service is not mentioned in the admin and user guide , and in install guide not clear too…. What a mess of oracle docs!
10gR3 ODDC install/admin and user guide
http://www.youtube.com/watch?v=4PwfGLw44oo a very good video – how to config the ODC (not ODDC) file storage etc.
https://www.youtube.com/watch?v=uYC-O0t30gc some help about the access db.
Only ODC support the The PDF (Searchable) option uses Optical Character Recognition (OCR) when creating PDF documents that users can search.
Note that the hidden text (the OCR results) within PDF (Searchable) documents CANNOT be edited and may contain errors.
Instead of access DB, you want to use the oracle DB, then you have to install the OLE driver for oracle, which is not in the software by default…I do not know why they do not like oracle DB.
By W.ZH Feb 2012
In my last article ESS-01050 error when you try to create a new scheduled process, I have shown one issue you may face in the scheduled process management; we can simply say the reason for that issue: Scheduled processes are confined by user and your role; you cannot see other user’s process. But system level, when you create a process, will detect the duplication process cross the user context. This is the reason you see that error message.
After you use the Scheduled processes page for a while you may find it have some limitations here. You can do create new process/resubmit/Put On Hold/Cancel/Release Process/View Log
Currently I do not try out on the Resubmit/Release Process feature yet, but I suppose they are not an update feature. We need some basic knowledge about the ESS here (Oracle Oracle Enterprise Scheduler Service). This scheduled process UI in fact use the ESS as the backend to trigger the job by the timer scheduled. For example, a daily job run at 5AM, when it will run tomorrow, at today 8AM, you will see its status is “Wait”, when it starts run at the time of tomorrow 5AM, you will see the status is “running”, after it is finished it normally shows “Succeeded”.
When it is in “wait” status, you can use the button to cancel it, then it will change to “cancelled” status, but as it is a daily job, so system will auto create a new job with default status “wait”. This new job scheduled to run at the day after tomorrow 5AM. So you must be very careful at here. As there is no update feature, such as, for this example, if you want to run this job today immediately, how can you do this?
You cannot create a new process with same task, as system does not allow duplicate task. You cannot change this current exist scheduler to let it can run immediately as system UI has no such feature. In fact, you stuck at here.
Because there is no change scheduler ability, at least I do not find it now.
You can use the EM to solve this issue, this need the WLS admin user account. All the Scheduled processes are the schedule jobs in the ESS server. Log into your EM for your domain management, under the ESS, you will see how to manage the scheduled jobs, each job with ID that can be matched to FA UI page. For a daily job, you can only change the schedule by going to the root job to change the whole schedule. Each daily is just a sub-job under the root job. I think I do not need to detail all the feature in the ESS admin ability, you need to refer the ESS user/dev guide for that.
How to save Streaming Media
– for FREE or for a small fee –
*** also see http://all-streaming-media.com/record-video-stream/
FLV Files (FLash Video) – FLV files are rather new but they have quickly become all the rage with these video clip sites where kids do dumb things so they can get hurt and show the world. FLV’s were created for streaming – but they can be delivered in three different ways:
Summary – How to grab the Various Types of streaming Video Files
*** FIRST *** always check to see if the site has provided a “Download” link, and if they have . . . right-click on the link and select “Save Target as . . .”
*** there is one utility called HiDownload that they claim can capture WMV, MOV, and RM files – however, no one I know has tried it !!!
Every now and then you come to a page with a fantastic flash movie, or a Real Audio song that you love, or a Windows Media video clip that is exactly what you have been looking for. No problem, you figure, “I’ll just right-click on it and save it like I do with images”. Woops, doesn’t work with streaming clips !!
Fortunately, the majority of these clips, while they stream in . . . are being saved to a file on your hard drive. This file is usually in the “Temporary Internet Files” folder. When the clip is finished loading, the file usually stays there until you move to another web page or close your browser.
Where is the “Temporary Internet Files” Folder ??
How to Save the Various Types of Streaming Media
see also http://www.streamingmediaworld.com
and for short (120 kB) ASF and RM sample clips – go to http://www.kidsmusiconline.com/sample.htm
the “Create your own Web Page” Trick – for Quicktime (*.mov files) and Real “*.RAM files”
If a link on a web page references a streaming video file directly, you just right-click on the link and select “Save Target as . . .” and then save the file. BUT quite often the link on the web page will play a video, but indirectly. It may take you to a page with the video clip embedded (Quicktime), or it may have a pointer text file, such as Real Audio/Video RAM and SMI files. Here is the trick:
An actual working sample of the “Create your own Web page” trick:
OK here is a link to an old, grainy video of my daughter: My daughter – I used a “ram” file, which is nothing more than a text file with a link to the actual clip, which is an rm file.
Test the file first, and play it, as you would on any website. Now suppose you want to save the actual clip, which is an rm file. You don’t know where the file resides, since the link does not go directly to that file. So do the following:
right-click on the link and select “Save Target as. . .” and then save the ram file
open the ram file (daughter.ram) in notepad – and you will see it has one line:
now, you can enter that line in your web browser’s address field and play the file – but you still cannot save it. So instead, create a web page from notepad, and save it as “c:\temp1.htm“. The web page will have the following text in it:
now just run the file, temp1.htm and it will open in your web browser and will have a single link in it to the rm file. Roght-click and select “Save Target as” to get the rm file. Give it any name you wish, such as “temp1.rm”. Oddly, IE will still tell you it is downloading a “ram” file, but it will actually be downloading the rm.
*** see http://www.flash-mx-template.com/ to buy cool custom Flash 5.0 templates that you can customize (you get the source code – the *.fla file) – or have them make a special template for you !! Of course, you need Flash MX to edit the fla file.
Easy to save – manually or using a flash capture utility. These are movies by Macromedia software – used for web page intro’s normally. The entire movie, which is an “swf” file, is saved before it begins playing. Simply search the “Temporary Internet Files” folder for *.swf, and then copy the file somewhere.
Flash Capture, by Dreamingsoft
This is an excellent flash capture utility. Once installed, for any flash movie that is playing, you just move your mouse over it and a tiny box pops up with an option to save the swf file wherever you choose !! They have a free 14-day trial, but it does not say the trial is disabled after that time – seems to just give a nag screen. There are several other similar utilities and as mentioned – Flash Movie Extract Pilot is a freebie.
How to embed your saved SWF File Into a Web Page
Just follow this sample code, which works for the Flash Movie file, M03L03P002.swf – simply replace that file name with your own, and adjust the display size (width and height) parameters, which appear in two places:
Background and the Difference between Flash and Shockwave
*** see also Flash vs Shockwave
Shockwave was created by Macromedia, to enable webbers to view Macromedia Director content directly in their browser. Macromedia also acquired and quickly updated and renamed the program to Flash 2, and the plug-in to Shockwave Flash Player. That, was when all the confusion began. Basically there are two players:
Shockwave usually refers to content created in Director and is used for interactive web add-ins, especially games. Flash refers to content created in Flash and is used mostly for movies, such as web page intros.
FLV files can be delivered in three different ways:
Easy to save manually. These are Windows Media Audio, Windows Media Video, Active Stream redirector and Advanced Systems Format. These are true streaming media files, in that they begin playing as soon as they have buffered (saved) enough data so that the clip can play without stopping in the middle. Of course, the algorithm that calculates how much to buffer before playing – makes a guess, which is often wrong, and the clip may stop several times to re-buffer. The point is, the file is not complete until you have played the entire clip!!!
The following method works for wmv, wma, asf, and asx files :
ASF and ASX
ASF is a streaming video or audio file – similar to Real Media rm files. It has extra capabilities such as slide shows and special controls.
ASX comes in 2 flavors. It was designed to be a Windows Metafile Extension, which is a text file that makes a call to the ASF file, similar to Real Media ram files – but it can also be a self-contained video or audio file.
Usually Impossible*** to save manually – requires a Real Media capture utility. These are also true streaming media files – unfortunately, only the pointer file is downloaded to Temporary Internet Files, and the actual media file is either held in mempry or stored with another filename.
RAM and SMI files – Real Media (*.rm and *.ra) files are not accessed directly by the link. For example, to play mysong.ra, the web page must include a ram file – that is a text reference (pointer), for example, mysong.ram. The ram file has one short line of text, with the URL of the Real Media clip. The web page will have a link to the ram file only.
Realmedia has gotten smart lately, and created SMI files to replace the older, limited RAM files. The smi file is an in-between link which rerouts the link yet again (therefore utilities such as xfileget, streamdpwn and streambox vcr only downloads the .smi file and gets fooled into thinking that’s the whole thing). “SMI” files are similar to RAM files, but they have a lot of additional features and parameters that can be added. Again, the SMI files merely points to the Real Media files, so you cannot download it from the link.
The line of text in the ram file begins with the protocol used to initiate streaming and can be
StreamDown utility (saves those “hard to get” Real Media files)
You can either buy ($39) or download the 15-day Trial of “CoCSoft Stream Down“. The trial is 15 days but see the modified date trick below. Or if that does not work, re-download the trial. It supports HTTP, FTP, RTSP, MMS, MMSU, and MMST. Simply paste the URL, and it downloads the associated media file !!
RAM only – DOES NOT WORK WITH SMI files !!! Similar to StreamDown – I tested this app and it did not work !!
Why the Manual Method of saving Real Media Files does not Work
Web browsers will open a URL with a ram file at the end and play the ra file in the Real Player – but they will not open a “page” with an ra or rm extension !!!
Now, it seems that if we could see the contents of the ram file we would see the URL to the ra file – but how to download it ?? Some say you can go to your browser’s file associations and remove the ra association – then your browser does not know how to play the file, and will ask you if you want to save it – but no, the browser will simply refuse to open that “page”. In addition, you typically cannot FTP files from a URL because the hosts do not permit FTP in general, except for specific special FTP locations.
The “gvp” file is NOT the video !!! It is merely a text file with the URL to the video inside of it. It also has other text to fool you – but here we show you what text to use for the URL. SIMPLE !!!
NOTE: “www.keepvids.com” used to be popular for saving Google videos – but Google changed something and keepvids it no longers works reliably – if at all !!!
DONE !!! The process looks cumbersome, but once you do it a couple of times it takes all of 10 seconds !!
You can now go back to the other IE window that is running Google Videos, and click the “Back” button to view and save more videos
Example of a Google gvp file and the actual URL of the Video
The gvp file contains the following text:
# download the free Google Video Player from http://video.google.com/
Here I have inserted dashes and arrows to show you where the text that you want begins and ends:
So when you select that text, it will be the following string:
1. Start Shell
For windows, essmsh.exe in the C:\Hyperion\AnalyticServices\bin or in the admin folder C:\Hyperion\AnalyticAdministrationServices\server\bin
essmsh -h | more to get help
essmsh -l user password to login
essmsh filename to use a file as input (can login in file)
Note: this user name is not the admin account for administration service, this account is account to start the essbase server, it is the essbase ‘s registered username
login username password on localhost;
5. spool on to ‘C:\esslog.log’; to log the input and output into a logfile, can use ‘\’
spool stdout on to ‘output.txt’;
spool stderr on to ‘errors.txt’;
6. spool off to stop the log
7. echo $ARBORPATH to display the path setting of essbase server
8. version; to see version of MaxL
9. logout; to loglout
10.exit; exit the shell
11. set var1 = sample; to set local variable
12. echo $var1; to see what the value of $var1 is
13. /* comments */ to comment
14. using $? to display last command’s return, 0 is ok, 1 is error in exec
0. ensure your DB for Essbase is running, it could be any DB(Oracle, mySQL, MSSQL, etc.) when you set up in you installation of essbase.
1. start the essbase server and sample in Hyperion–>essbase, you need input the server password to start it.
2. start the administration service server in Hyperion–>administration service
3. start the administration service console in Hyperion–>administration service, input your admin account and password.
4. then can start use it and monitor it.